| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
on x86 and some other archs, functions which make function calls which
might go through a PLT incur a significant overhead cost loading the
GOT register prior to making the call. this load is utterly useless in
musl, since all calls are bound at library-creation time using
-Bsymbolic-functions, but the compiler has no way of knowing this, and
attempts to set the default visibility to protected have failed due to
bugs in GCC and binutils.
this commit simply manually assigns hidden/protected visibility, as
appropriate, to a few internal-use-only functions which have many
callers, or which have callers that are hot paths like getc/putc. it
shaves about 5k off the i386 libc.so with -Os. many of the
improvements are in syscall wrappers, where the benefit is just size
and performance improvement is unmeasurable noise amid the syscall
overhead. however, stdio may be measurably faster.
if in the future there are toolchains that can do the same thing
globally without introducing linking bugs, it might be worth
considering removing these workarounds.
|
|
these functions must behave as if they obtain the lock via flockfile
to satisfy POSIX requirements. since another thread can provably hold
the lock when they are called, they must wait to obtain the lock
before they can return, even if the correct return value could be
obtained without locking. in the case of fclose and freopen, failure
to do so could cause correct (albeit obscure) programs to crash or
otherwise misbehave; in the case of feof, ferror, and fwide, failure
to obtain the lock could sometimes return incorrect results. in any
case, having these functions proceed and return while another thread
held the lock was wrong.
|
|
1. don't open /dev/null just as a basis to copy flags; use shared
__fmodeflags function to get the right file flags for the mode.
2. handle the case (probably invalid, but whatever) case where the
original stream's file descriptor was closed; previously, the logic
re-closed it.
3. accept the "e" mode flag for close-on-exec; update dup3 to fallback
to using dup2 so we can simply call __dup3 instead of putting fallback
logic in freopen itself.
|
|
|
|
gcc seems to be generating identical or near-identical code for both
versions, but the newer code is more expressive of what it's doing.
|
|
the behavior of putenv is left undefined if the argument does not
contain an equal sign, but traditional implementations behave this way
and gnulib replaces putenv if it doesn't do this.
|
|
this will prevent gnulib from wrapping our strtod to handle this
useless feature.
|
|
signal mask was not being restored after fork, but instead blocked again.
|
|
|
|
this was broken during the early dynamic-linked TLS commits, which
rearranged some of the code for handling new relocation types.
|
|
__release_ptc() is only valid in the parent; if it's performed in the
child, the lock will be unlocked early then double-unlocked later,
corrupting the lock state.
|
|
|
|
with this commit, based on testing with patches to qemu which are not
yet upstream,
|
|
since it did not set the return-value register, the caller could
wrongly interpret this as failure.
|
|
|
|
|
|
|
|
since we target systems without overcommit, special care should be
taken that system() and popen(), like posix_spawn(), do not fail in
processes whose commit charges are too high to allow ordinary forking.
this in turn requires special precautions to ensure that the parent
process's signal handlers do not end up running in the shared-memory
child, where they could corrupt the state of the parent process.
popen has also been updated to use pipe2, so it does not have a
fd-leak race in multi-threaded programs. since pipe2 is missing on
older kernels, (non-atomic) emulation has been added.
some silly bugs in the old code should be gone too.
|
|
|
|
only @PLT relocations are considered functions for purposes of
-Bsymbolic-functions, so always use @PLT. it should not hurt in the
static-linked case.
|
|
based on strstr. passes gnulib tests and a few quick checks of my own.
|
|
despite documentation that makes it sound a lot different, the only
ABI-constraint difference between TLS variants II and I seems to be
that variant II stores the initial TLS segment immediately below the
thread pointer (i.e. the thread pointer points to the end of it) and
variant I stores the initial TLS segment above the thread pointer,
requiring the thread descriptor to be stored below. the actual value
stored in the thread pointer register also tends to have per-arch
random offsets applied to it for silly micro-optimization purposes.
with these changes applied, TLS should be basically working on all
supported archs except microblaze. I'm still working on getting the
necessary information and a working toolchain that can build TLS
binaries for microblaze, but in theory, static-linked programs with
TLS and dynamic-linked programs where only the main executable uses
TLS should already work on microblaze.
alignment constraints have not yet been heavily tested, so it's
possible that this code does not always align TLS segments correctly
on archs that need TLS variant I.
|
|
usage of vfork creates a situation where a process of lower privilege
may momentarily have write access to the memory of a process of higher
privilege.
consider the case of a multi-threaded suid program which is calling
posix_spawn in one thread while another thread drops the elevated
privileges then runs untrusted (relative to the elevated privilege)
code as the original invoking user. this untrusted code can then
potentially modify the data the child process will use before calling
exec, for example changing the pathname or arguments that will be
passed to exec.
note that if vfork is implemented as fork, the lock will not be held
until the child execs, but since memory is not shared it does not
matter.
|
|
|
|
this change brings the behavior in line with the static-linked code,
which seems to be correct.
|
|
with this change, pcc-built musl libc.so seems to work correctly. the
problem is that pcc generates GOT lookups for external-linkage symbols
even if they are hidden, rather than using GOT-relative addressing.
the entire reason we're using hidden visibility on the __libc object
is to make it accessible prior to relocations -- not to mention
inexpensive to access. unfortunately, the workaround makes it even
more expensive on pcc.
when the pcc issue is fixed, an appropriate version test should be
added so new pcc can use the much more efficient variant.
|
|
this makes it so the #undef libc and __libc name are no longer needed,
which were problematic because the "accessor function" mode for
accessing the libc struct could not be used, breaking build on any
compiler without (working) visibility.
|
|
|
|
|
|
this is necessary because posix_spawn calls sigaction after vfork, and
if the thread pointer is not already initialized, initializing it in
the child corrupts the parent process's state.
|
|
this doubles the performance of the fastest syscalls on the atom I
tested it on; improvement is reportedly much more dramatic on
worst-case cpus. cannot be used for cancellable syscalls.
|
|
|
|
the code in __libc_start_main is now responsible for parsing auxv,
rather than duplicating the parsing all over the place. this should
shave off a few cycles and some code size. __init_libc is left as an
external-linkage function despite the fact that it could be static, to
prevent it from being inlined and permanently wasting stack space when
main is called.
a few other minor changes are included, like eliminating per-thread
ssp canaries (they were likely broken when combined with certain
dlopen usages, and completely unnecessary) and some other unnecessary
checks. since this code gets linked into every program, it should be
as small and simple as possible.
|
|
at initial program load, all libraries must be loaded before the
thread pointer can be setup, since the TP-relative addresses of all
initial TLS objects must be constant.
|
|
this is needed to ensure async-cancel-safety, i.e. to make it safe to
access TLS objects when async cancellation is enabled. otherwise, if
cancellation were acter upon after the atomic fetch/add but before the
thread saved the obtained memory, another access to the same TLS in
the cancellation handler could end up performing the atomic fetch/add
again, consuming more memory than is actually available and
overflowing into other objects on the heap.
|
|
|
|
|
|
symbol value of 0 is not "undefined" for TLS; it's the address of the
first symbol in the TLS segment. however, non-definition TLS
references also have values of 0, so check the section.
hopefully the new logic is more clear, too.
|
|
compute offsets from the thread pointer statically when loading the
library, rather than repeating the logic on each thread creation. not
only is the latter less efficient at runtime; it also fails to provide
solid guarantees that the offsets will remain the same when the
initial alignment of memory is different. the new alignment handling
is both more rigorous and simpler.
the old code was also clobbering TLS bss with random image data in
some cases due to using tls_size (size of TLS segment) instead of
tls_len (length of the TLS data image).
|
|
some libraries call dlopen from their constructors, resulting in
recursive calls to dlopen. previously, this resulted in deadlock. I'm
now unlocking the dlopen lock before running constructors (this is
especially important since the lock also blocked pthread_create and
was being held while application code runs!) and using a separate
recursive mutex protecting the ctor/dtor state instead.
in order to prevent the same ctor from being called more than once, a
module is considered "constructed" just before the ctor runs.
also, switch from using atexit to register each dtor to using a single
atexit call to register the dynamic linker's dtor processing as just
one handler. this is necessary because atexit performs allocation and
may fail, but the library has already been loaded and cannot be
backed-out at the time dtor registration is performed. this change
also ensures that all dtors run after all atexit functions, rather
than in mixed order.
|
|
libraries loaded more than once by pathname should not get shortnames
that would cause them to later be used to satisfy non-pathname load
requests.
|
|
unlike other implementations, this one reserves memory for new TLS in
all pre-existing threads at dlopen-time, and dlopen will fail with no
resources consumed and no new libraries loaded if memory is not
available. memory is not immediately distributed to running threads;
that would be too complex and too costly. instead, assurances are made
that threads needing the new TLS can obtain it in an async-signal-safe
way from a buffer belonging to the dynamic linker/new module (via
atomic fetch-and-add based allocator).
I've re-appropriated the lock that was previously used for __synccall
(synchronizing set*id() syscalls between threads) as a general
pthread_create lock. it's a "backwards" rwlock where the "read"
operation is safe atomic modification of the live thread count, which
multiple threads can perform at the same time, and the "write"
operation is making sure the count does not increase during an
operation that depends on it remaining bounded (__synccall or dlopen).
in static-linked programs that don't use __synccall, this lock is a
no-op and has no cost.
|
|
orig_tail was being saved before the lock was obtained, allowing
dlopen failure to roll-back other dlopens that had succeeded.
|
|
currently, only i386 is tested. x86_64 and arm should probably work.
the necessary relocation types for mips and microblaze have not been
added because I don't understand how they're supposed to work, and I'm
not even sure if it's defined yet on microblaze. I may be able to
reverse engineer the requirements out of gcc/binutils output.
|
|
this was an optimization to save/recover a minimal amount of extra
memory for use by malloc, that's becoming increasingly costly to keep
around. freeing this data:
1. breaks debugging with gdb (it can't find library symbols)
2. breaks thread-local storage in shared libraries
it would be possible to disable freeing when TLS is used, but in
addition to the above breakages, tracking whether dlopen/dlsym is used
adds a cost to every symbol lookup, possibly making program startup
slower for large programs. combined with the complexity, it's not
worth it. we already save/recover plenty of memory in the dynamic
linker with reclaim_gaps.
|
|
this code will not work yet because the necessary relocations are not
supported, and cannot be supported without some internal changes to
how relocation processing works (coming soon).
|
|
only TLS in the main program is supported so far; TLS defined in
shared libraries will not work yet.
|
|
the design for TLS in dynamic-linked programs is mostly complete too,
but I have not yet implemented it. cost is nonzero but still low for
programs which do not use TLS and/or do not use threads (a few hundred
bytes of new code, plus dependency on memcpy). i believe it can be
made smaller at some point by merging __init_tls and __init_security
into __libc_start_main and avoiding duplicate auxv-parsing code.
at the same time, I've also slightly changed the logic pthread_create
uses to allocate guard pages to ensure that guard pages are not
counted towards commit charge.
|
|
based on proposed patches by Daniel Cegiełka, with minor changes:
- use a weak symbol for optreset so it doesn't clash with namespace
- also reset optpos (position in multi-option arg like -lR)
- also make getopt_long support reset
|
|
also fix one minor bug: failure to free the early-reserved slot when
the semaphore later found to already be mapped.
|
