fix failure to obtain EOWNERDEAD status for process-shared robust mutexes

Linux's documentation (robust-futex-ABI.txt) claims that, when a
process dies with a futex on the robust list, bit 30 (0x40000000) is
set to indicate the status. however, what actually happens is that
bits 0-30 are replaced with the value 0x40000000, i.e. bits 0-29
(containing the old owner tid) are cleared at the same time bit 30 is
set.

our userspace-side code for robust mutexes was written based on that
documentation, assuming that kernel would never produce a futex value
of 0x40000000, since the low (owner) bits would always be non-zero.
commit d338b506e39b1e2c68366b12be90704c635602ce introduced this
assumption explicitly while fixing another bug in how non-recoverable
status for robust mutexes was tracked. presumably the tests conducted
at that time only checked non-process-shared robust mutexes, which are
handled in pthread_exit (which implemented the documented kernel
protocol, not the actual one) rather than by the kernel.

change pthread_exit robust list processing to match the kernel
behavior, clearing bits 0-29 while setting bit 30, and use the value
0x7fffffff instead of 0x40000000 to encode non-recoverable status. the
choice of value here is arbitrary; any value with at least one of bits
0-29 set should work just as well,

3 files changed, 3 insertions, 3 deletions

diff --git a/src/thread/pthread_create.c b/src/thread/pthread_create.c
index e7df34a9..9f6b98e6 100644
--- a/src/thread/pthread_create.c
+++ b/src/thread/pthread_create.c
@@ -79,7 +79,7 @@ _Noreturn void __pthread_exit(void *result)
 		int priv = (m->_m_type & 128) ^ 128;
 		self->robust_list.pending = rp;
 		self->robust_list.head = *rp;
-		int cont = a_swap(&m->_m_lock, self->tid|0x40000000);
+		int cont = a_swap(&m->_m_lock, 0x40000000);
 		self->robust_list.pending = 0;
 		if (cont < 0 || waiters)
 			__wake(&m->_m_lock, 1, priv);
diff --git a/src/thread/pthread_mutex_trylock.c b/src/thread/pthread_mutex_trylock.c
index 0df3ce29..54876a61 100644
--- a/src/thread/pthread_mutex_trylock.c
+++ b/src/thread/pthread_mutex_trylock.c
@@ -14,7 +14,7 @@ int __pthread_mutex_trylock_owner(pthread_mutex_t *m)
 		m->_m_count++;
 		return 0;
 	}
-	if (own == 0x40000000) return ENOTRECOVERABLE;
+	if (own == 0x7fffffff) return ENOTRECOVERABLE;
 
 	if (m->_m_type & 128) {
 		if (!self->robust_list.off) {
diff --git a/src/thread/pthread_mutex_unlock.c b/src/thread/pthread_mutex_unlock.c
index 02da92a9..7dd00d27 100644
--- a/src/thread/pthread_mutex_unlock.c
+++ b/src/thread/pthread_mutex_unlock.c
@@ -24,7 +24,7 @@ int __pthread_mutex_unlock(pthread_mutex_t *m)
 		if (next != &self->robust_list.head) *(volatile void *volatile *)
 			((char *)next - sizeof(void *)) = prev;
 	}
-	cont = a_swap(&m->_m_lock, (type & 8) ? 0x40000000 : 0);
+	cont = a_swap(&m->_m_lock, (type & 8) ? 0x7fffffff : 0);
 	if (type != PTHREAD_MUTEX_NORMAL && !priv) {
 		self->robust_list.pending = 0;
 		__vm_unlock();