summaryrefslogtreecommitdiff
path: root/src/internal/libc.h
AgeCommit message (Collapse)AuthorFilesLines
2013-09-20fix potential deadlock bug in libc-internal locking logicRich Felker1-2/+2
if a multithreaded program became non-multithreaded (i.e. all other threads exited) while one thread held an internal lock, the remaining thread would fail to release the lock. the the program then became multithreaded again at a later time, any further attempts to obtain the lock would deadlock permanently. the underlying cause is that the value of libc.threads_minus_1 at unlock time might not match the value at lock time. one solution would be returning a flag to the caller indicating whether the lock was taken and needs to be unlocked, but there is a simpler solution: using the lock itself as such a flag. note that this flag is not needed anyway for correctness; if the lock is not held, the unlock code is harmless. however, the memory synchronization properties associated with a_store are costly on some archs, so it's best to avoid executing the unlock code when it is unnecessary.
2013-09-15support configurable page size on mips, powerpc and microblazeSzabolcs Nagy1-0/+6
PAGE_SIZE was hardcoded to 4096, which is historically what most systems use, but on several archs it is a kernel config parameter, user space can only know it at execution time from the aux vector. PAGE_SIZE and PAGESIZE are not defined on archs where page size is a runtime parameter, applications should use sysconf(_SC_PAGE_SIZE) to query it. Internally libc code defines PAGE_SIZE to libc.page_size, which is set to aux[AT_PAGESZ] in __init_libc and early in __dynlink as well. (Note that libc.page_size can be accessed without GOT, ie. before relocations are done) Some fpathconf settings are hardcoded to 4096, these should be actually queried from the filesystem using statfs.
2013-07-21add support for init/fini array in main program, and greatly simplifyRich Felker1-3/+0
modern (4.7.x and later) gcc uses init/fini arrays, rather than the legacy _init/_fini function pasting and crtbegin/crtend ctors/dtors system, on most or all archs. some archs had already switched a long time ago. without following this change, global ctors/dtors will cease to work under musl when building with new gcc versions. the most surprising part of this patch is that it actually reduces the size of the init code, for both static and shared libc. this is achieved by (1) unifying the handling main program and shared libraries in the dynamic linker, and (2) eliminating the glibc-inspired rube goldberg machine for passing around init and fini function pointers. to clarify, some background: the function signature for __libc_start_main was based on glibc, as part of the original goal of being able to run some glibc-linked binaries. it worked by having the crt1 code, which is linked into every application, static or dynamic, obtain and pass pointers to the init and fini functions, which __libc_start_main is then responsible for using and recording for later use, as necessary. however, in neither the static-linked nor dynamic-linked case do we actually need crt1.o's help. with dynamic linking, all the pointers are available in the _DYNAMIC block. with static linking, it's safe to simply access the _init/_fini and __init_array_start, etc. symbols directly. obviously changing the __libc_start_main function signature in an incompatible way would break both old musl-linked programs and glibc-linked programs, so let's not do that. instead, the function can just ignore the information it doesn't need. new archs need not even provide the useless args in their versions of crt1.o. existing archs should continue to provide it as long as there is an interest in having newly-linked applications be able to run on old versions of musl; at some point in the future, this support can be removed.
2013-02-17consistently use the internal name __environ for environRich Felker1-1/+0
patch by Jens Gustedt. previously, the intended policy was to use __environ in code that must conform to the ISO C namespace requirements, and environ elsewhere. this policy was not followed in practice anyway, making things confusing. on top of that, Jens reported that certain combinations of link-time optimization options were breaking with the inconsistent references; this seems to be a compiler or linker bug, but having it go away is a nice side effect of the changes made here.
2012-12-07fix trailing whitespace issues that crept in here and thereRich Felker1-1/+1
2012-10-25use explicit visibility to optimize a few hot-path function callsRich Felker1-4/+4
on x86 and some other archs, functions which make function calls which might go through a PLT incur a significant overhead cost loading the GOT register prior to making the call. this load is utterly useless in musl, since all calls are bound at library-creation time using -Bsymbolic-functions, but the compiler has no way of knowing this, and attempts to set the default visibility to protected have failed due to bugs in GCC and binutils. this commit simply manually assigns hidden/protected visibility, as appropriate, to a few internal-use-only functions which have many callers, or which have callers that are hot paths like getc/putc. it shaves about 5k off the i386 libc.so with -Os. many of the improvements are in syscall wrappers, where the benefit is just size and performance improvement is unmeasurable noise amid the syscall overhead. however, stdio may be measurably faster. if in the future there are toolchains that can do the same thing globally without introducing linking bugs, it might be worth considering removing these workarounds.
2012-10-13workaround broken hidden-visibility handling in pccRich Felker1-1/+1
with this change, pcc-built musl libc.so seems to work correctly. the problem is that pcc generates GOT lookups for external-linkage symbols even if they are hidden, rather than using GOT-relative addressing. the entire reason we're using hidden visibility on the __libc object is to make it accessible prior to relocations -- not to mention inexpensive to access. unfortunately, the workaround makes it even more expensive on pcc. when the pcc issue is fixed, an appropriate version test should be added so new pcc can use the much more efficient variant.
2012-10-05support for TLS in dynamic-loaded (dlopen) modulesRich Felker1-1/+1
unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-04TLS (GNU/C11 thread-local storage) support for static-linked programsRich Felker1-0/+1
the design for TLS in dynamic-linked programs is mostly complete too, but I have not yet implemented it. cost is nonzero but still low for programs which do not use TLS and/or do not use threads (a few hundred bytes of new code, plus dependency on memcpy). i believe it can be made smaller at some point by merging __init_tls and __init_security into __libc_start_main and avoiding duplicate auxv-parsing code. at the same time, I've also slightly changed the logic pthread_create uses to allocate guard pages to ensure that guard pages are not counted towards commit charge.
2012-07-27save AT_HWCAP from auxv for subsequent use in machine-specific codeRich Felker1-0/+1
it's expected that this will be needed/useful only in asm, so I've given it its own symbol that can be addressed in pc-relative ways from asm rather than adding a field in the __libc structure which would require hard-coding the offset wherever it's used.
2012-05-31enable LARGEFILE64 aliasesRich Felker1-2/+1
these will NOT be used when compiling with -D_LARGEFILE64_SOURCE on musl; instead, they exist in the hopes of eventually being able to run some glibc-linked apps with musl sitting in place of glibc. also remove the (apparently incorrect) fcntl alias.
2012-05-22remove everything related to forkallRich Felker1-1/+0
i made a best attempt, but the intended semantics of this function are fundamentally contradictory. there is no consistent way to handle ownership of locks when forking a multi-threaded process. the code could have worked by accident for programs that only used normal mutexes and nothing else (since they don't actually store or care about their owner), but that's about it. broken-by-design interfaces that aren't even in glibc (only solaris) don't belong in musl.
2012-04-24ditch the priority inheritance locks; use malloc's version of lockRich Felker1-1/+1
i did some testing trying to switch malloc to use the new internal lock with priority inheritance, and my malloc contention test got 20-100 times slower. if priority inheritance futexes are this slow, it's simply too high a price to pay for avoiding priority inversion. maybe we can consider them somewhere down the road once the kernel folks get their act together on this (and perferably don't link it to glibc's inefficient lock API)... as such, i've switch __lock to use malloc's implementation of lightweight locks, and updated all the users of the code to use an array with a waiter count for their locks. this should give optimal performance in the vast majority of cases, and it's simple. malloc is still using its own internal copy of the lock code because it seems to yield measurably better performance with -O3 when it's inlined (20% or more difference in the contention stress test).
2012-04-24new internal locking primitive; drop spinlocksRich Felker1-1/+2
we use priority inheritance futexes if possible so that the library cannot hit internal priority inversion deadlocks in the presence of realtime priority scheduling (full support to be added later).
2012-02-24new attempt at working around the gcc 3 visibility bugRich Felker1-0/+3
since gcc is failing to generate the necessary ".hidden" directive in the output asm, generate it explicitly with an __asm__ statement...
2012-02-23cleanup and work around visibility bug in gcc 3 that affects x86_64Rich Felker1-5/+10
in gcc 3, the visibility attribute must be placed on both the declaration and on the definition. if it's omitted from the definition, the compiler fails to emit the ".hidden" directive in the assembly, and the linker will either generate textrels (if supported, such as on i386) or refuse to link (on targets where certain types of textrels are forbidden or impossible without further assumptions about memory layout, such as on x86_64). this patch also unifies the decision about when to use visibility into libc.h and makes the visibility in the utf-8 state machine tables based on libc.h rather than a duplicate test.
2011-08-23security hardening: ensure suid programs have valid stdin/out/errRich Felker1-2/+4
this behavior (opening fds 0-2 for a suid program) is explicitly allowed (but not required) by POSIX to protect badly-written suid programs from clobbering files they later open. this commit does add some cost in startup code, but the availability of auxv and the security flag will be useful elsewhere in the future. in particular auxv is needed for static-linked vdso support, which is still waiting to be committed (sorry nik!)
2011-08-12pthread and synccall cleanup, new __synccall_wait opRich Felker1-0/+1
fix up clone signature to match the actual behavior. the new __syncall_wait function allows a __synccall callback to wait for other threads to continue without returning, so that it can resume action after the caller finishes. this interface could be made significantly more general/powerful with minimal effort, but i'll wait to do that until it's actually useful for something.
2011-08-06simplify multi-threaded errno, eliminate useless function pointerRich Felker1-2/+1
2011-08-06use weak aliases rather than function pointers to simplify some codeRich Felker1-2/+0
2011-07-30add proper fuxed-based locking for stdioRich Felker1-1/+2
previously, stdio used spinlocks, which would be unacceptable if we ever add support for thread priorities, and which yielded pathologically bad performance if an application attempted to use flockfile on a key file as a major/primary locking mechanism. i had held off on making this change for fear that it would hurt performance in the non-threaded case, but actually support for recursive locking had already inflicted that cost. by having the internal locking functions store a flag indicating whether they need to perform unlocking, rather than using the actual recursive lock counter, i was able to combine the conditionals at unlock time, eliminating any additional cost, and also avoid a nasty corner case where a huge number of calls to ftrylockfile could cause deadlock later at the point of internal locking. this commit also fixes some issues with usage of pthread_self conflicting with __attribute__((const)) which resulted in crashes with some compiler versions/optimizations, mainly in flockfile prior to pthread_create.
2011-07-29new attempt at making set*id() safe and robustRich Felker1-1/+2
changing credentials in a multi-threaded program is extremely difficult on linux because it requires synchronizing the change between all threads, which have their own thread-local credentials on the kernel side. this is further complicated by the fact that changing the real uid can fail due to exceeding RLIMIT_NPROC, making it possible that the syscall will succeed in some threads but fail in others. the old __rsyscall approach being replaced was robust in that it would report failure if any one thread failed, but in this case, the program would be left in an inconsistent state where individual threads might have different uid. (this was not as bad as glibc, which would sometimes even fail to report the failure entirely!) the new approach being committed refuses to change real user id when it cannot temporarily set the rlimit to infinity. this is completely POSIX conformant since POSIX does not require an implementation to allow real-user-id changes for non-privileged processes whatsoever. still, setting the real uid can fail due to memory allocation in the kernel, but this can only happen if there is not already a cached object for the target user. thus, we forcibly serialize the syscalls attempts, and fail the entire operation on the first failure. this *should* lead to an all-or-nothing success/failure result, but it's still fragile and highly dependent on kernel developers not breaking things worse than they're already broken. ideally linux will eventually add a CLONE_USERCRED flag that would give POSIX conformant credential changes without any hacks from userspace, and all of this code would become redundant and could be removed ~10 years down the line when everyone has abandoned the old broken kernels. i'm not holding my breath...
2011-04-20fix minor bugs due to incorrect threaded-predicate semanticsRich Felker1-0/+1
some functions that should have been testing whether pthread_self() had been called and initialized the thread pointer were instead testing whether pthread_create() had been called and actually made the program "threaded". while it's unlikely any mismatch would occur in real-world problems, this could have introduced subtle bugs. now, we store the address of the main thread's thread descriptor in the libc structure and use its presence as a flag that the thread register is initialized. note that after fork, the calling thread (not necessarily the original main thread) is the new main thread.
2011-04-17clean up handling of thread/nothread mode, lockingRich Felker1-4/+3
2011-04-17optimize cancellation enable/disable codeRich Felker1-0/+1
the goal is to be able to use pthread_setcancelstate internally in the implementation, whenever a function might want to use functions which are cancellation points but avoid becoming a cancellation point itself. i could have just used a separate internal function for temporarily inhibiting cancellation, but the solution in this commit is better because (1) it's one less implementation-specific detail in functions that need to use it, and (2) application code can also get the same benefit. previously, pthread_setcancelstate dependend on pthread_self, which would pull in unwanted thread setup overhead for non-threaded programs. now, it temporarily stores the state in the global libc struct if threads have not been initialized, and later moves it if needed. this way we can instead use __pthread_self, which has no dependencies and assumes that the thread register is already valid.
2011-04-17overhaul pthread cancellationRich Felker1-7/+1
this patch improves the correctness, simplicity, and size of cancellation-related code. modulo any small errors, it should now be completely conformant, safe, and resource-leak free. the notion of entering and exiting cancellation-point context has been completely eliminated and replaced with alternative syscall assembly code for cancellable syscalls. the assembly is responsible for setting up execution context information (stack pointer and address of the syscall instruction) which the cancellation signal handler can use to determine whether the interrupted code was in a cancellable state. these changes eliminate race conditions in the previous generation of cancellation handling code (whereby a cancellation request received just prior to the syscall would not be processed, leaving the syscall to block, potentially indefinitely), and remedy an issue where non-cancellable syscalls made from signal handlers became cancellable if the signal handler interrupted a cancellation point. x86_64 asm is untested and may need a second try to get it right.
2011-04-06move rsyscall out of pthread_create moduleRich Felker1-1/+2
this is something of a tradeoff, as now set*id() functions, rather than pthread_create, are what pull in the code overhead for dealing with linux's refusal to implement proper POSIX thread-vs-process semantics. my motivations are: 1. it's cleaner this way, especially cleaner to optimize out the rsyscall locking overhead from pthread_create when it's not needed. 2. it's expected that only a tiny number of core system programs will ever use set*id() functions, whereas many programs may want to use threads, and making thread overhead tiny is an incentive for "light" programs to try threads.
2011-04-05new framework to inhibit thread cancellation when neededRich Felker1-0/+2
with these small changes, libc functions which need to call functions which are cancellation points, but which themselves must not be cancellation points, can use the CANCELPT_INHIBIT and CANCELPT_RESUME macros to temporarily inhibit all cancellation.
2011-04-03simplify calling of timer signal handlerRich Felker1-2/+1
2011-04-03simplify pthread tsd key handlingRich Felker1-2/+1
2011-04-01reorganize the __libc structure for threaded performance issuesRich Felker1-6/+6
we want to keep atomically updated fields (locks and thread count) and really anything writable far away from frequently-needed function pointers. stuff some rarely-needed function pointers in between to pad, hopefully up to a cache line boundary.
2011-03-29major improvements to cancellation handlingRich Felker1-0/+1
- there is no longer any risk of spoofing cancellation requests, since the cancel flag is set in pthread_cancel rather than in the signal handler. - cancellation signal is no longer unblocked when running the cancellation handlers. instead, pthread_create will cause any new threads created from a cancellation handler to unblock their own cancellation signal. - various tweaks in preparation for POSIX timer support.
2011-03-24overhaul cancellation to fix resource leaks and dangerous behavior with signalsRich Felker1-1/+2
this commit addresses two issues: 1. a race condition, whereby a cancellation request occurring after a syscall returned from kernelspace but before the subsequent CANCELPT_END would cause cancellable resource-allocating syscalls (like open) to leak resources. 2. signal handlers invoked while the thread was blocked at a cancellation point behaved as if asynchronous cancellation mode wer in effect, resulting in potentially dangerous state corruption if a cancellation request occurs. the glibc/nptl implementation of threads shares both of these issues. with this commit, both are fixed. however, cancellation points encountered in a signal handler will not be acted upon if the signal was received while the thread was already at a cancellation point. they will of course be acted upon after the signal handler returns, so in real-world usage where signal handlers quickly return, it should not be a problem. it's possible to solve this problem too by having sigaction() wrap all signal handlers with a function that uses a pthread_cleanup handler to catch cancellation, patch up the saved context, and return into the cancellable function that will catch and act upon the cancellation. however that would be a lot of complexity for minimal if any benefit...
2011-03-12implement flockfile api, rework stdio lockingRich Felker1-0/+4
2011-02-24various changes in preparation for dynamic linking supportRich Felker1-4/+12
prefer using visibility=hidden for __libc internal data, rather than an accessor function, if the compiler has visibility. optimize with -O3 for PIC targets (shared library). without heavy inlining, reloading the GOT register in small functions kills performance. 20-30% size increase for a single libc.so is not a big deal, compared to comparaible size increase in every static binaries. use -Bsymbolic-functions, not -Bsymbolic. global variables are subject to COPY relocations, and thus binding their addresses in the library at link time will cause library functions to read the wrong (original) copies instead of the copies made in the main program's bss section. add entry point, _start, for dynamic linker.
2011-02-20use an accessor function for __libc data pointer when compiled as PICRich Felker1-3/+10
prior to this change, a large portion of libc was unusable prior to relocation by the dynamic linker, due to dependence on the global data in the __libc structure and the need to obtain its address through the GOT. with this patch, the accessor function __libc_loc is now able to obtain the address of __libc via PC-relative addressing without using the GOT. this means the majority of libc functionality is now accessible right away. naturally, the above statements all depend on having an architecture where PC-relative addressing and jumps/calls are feasible, and a compiler that generates the appropriate code.
2011-02-18add pthread_atfork interfaceRich Felker1-0/+1
note that this presently does not handle consistency of the libc's own global state during forking. as per POSIX 2008, if the parent process was threaded, the child process may only call async-signal-safe functions until one of the exec-family functions is called, so the current behavior is believed to be conformant even if non-ideal. it may be improved at some later time.
2011-02-12initial check-in, version 0.5.0v0.5.0Rich Felker1-0/+43