summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)AuthorFilesLines
2015-06-28dynlink.c: slim down gnu_lookupAlexander Monakov1-9/+5
Do not reference dso->syms and dso->strings until point of use. Check 'h1 == (h2|1)', the simplest condition, before the others.
2015-06-28dynlink.c: use bloom filter in gnu hash lookupAlexander Monakov1-3/+22
Introduce gnu_lookup_filtered and use it to speed up symbol lookups in find_sym (do_dlsym is left as is, based on an expectation that frequently dlsym queries will use a dlopen handle rather than RTLD_NEXT or RTLD_DEFAULT, and will not need to look at more than one DSO).
2015-06-27dynlink.c: use a faster expression in gnu_hashAlexander Monakov1-1/+1
With -Os, GCC uses a multiply rather than a shift and addition for 'h*33'. Use a more efficient expression explicitely.
2015-06-25fix local-dynamic model TLS on mips and powerpcRich Felker3-5/+9
the TLS ABI spec for mips, powerpc, and some other (presently unsupported) RISC archs has the return value of __tls_get_addr offset by +0x8000 and the result of DTPOFF relocations offset by -0x8000. I had previously assumed this part of the ABI was actually just an implementation detail, since the adjustments cancel out. however, when the local dynamic model is used for accessing TLS that's known to be in the same DSO, either of the following may happen: 1. the -0x8000 offset may already be applied to the argument structure passed to __tls_get_addr at ld time, without any opportunity for runtime relocations. 2. __tls_get_addr may be used with a zero offset argument to obtain a base address for the module's TLS, to which the caller then applies immediate offsets for individual objects accessed using the local dynamic model. since the immediate offsets have the -0x8000 adjustment applied to them, the base address they use needs to include the +0x8000 offset. it would be possible, but more complex, to store the pointers in the dtv[] array with the +0x8000 offset pre-applied, to avoid the runtime cost of adding 0x8000 on each call to __tls_get_addr. this change could be made later if measurements show that it would help.
2015-06-23make dynamic linker work around MAP_FAILED mmap failure on nommu kernelsRich Felker1-2/+24
previously, loading of additional libraries beyond libc/ldso did not work on nommu kernels, nor did loading programs via invocation of the dynamic linker as a command.
2015-06-23reimplement strverscmp to fix corner casesRich Felker1-32/+25
this interface is non-standardized and is a GNU invention, and as such, our implementation should match the behavior of the GNU function. one peculiarity the old implementation got wrong was the handling of all-zero digit sequences: they are supposed to compare greater than digit sequences of which they are a proper prefix, as in 009 < 00. in addition, high bytes were treated with char signedness rather than as unsigned. this was wrong regardless of what the GNU function does since the resulting order relation varied by arch. the new strverscmp implementation makes explicit the cases where the order differs from what strcmp would produce, of which there are only two.
2015-06-22fix regression/typo that disabled __simple_malloc when calloc is usedRich Felker1-1/+1
commit ba819787ee93ceae94efd274f7849e317c1bff58 introduced this regression. since the __malloc0 weak alias was not properly provided by __simple_malloc, use of calloc forced the full malloc to be linked.
2015-06-22fix calloc when __simple_malloc implementation is usedRich Felker3-12/+15
previously, calloc's implementation encoded assumptions about the implementation of malloc, accessing a size_t word just prior to the allocated memory to determine if it was obtained by mmap to optimize out the zero-filling. when __simple_malloc is used (static linking a program with no realloc/free), it doesn't matter if the result of this check is wrong, since all allocations are zero-initialized anyway. but the access could be invalid if it crosses a page boundary or if the pointer is not sufficiently aligned, which can happen for very small allocations. this patch fixes the issue by moving the zero-fill logic into malloc.c with the full malloc, as a new function named __malloc0, which is provided by a weak alias to __simple_malloc (which always gives zero-filled memory) when the full malloc is not in use.
2015-06-20provide __stack_chk_fail_local in libc.aRich Felker1-0/+4
this symbol is needed only on archs where the PLT call ABI is klunky, and only for position-independent code compiled with stack protector. thus references usually only appear in shared libraries or PIE executables, but they can also appear when linking statically if some of the object files being linked were built as PIC/PIE. normally libssp_nonshared.a from the compiler toolchain should provide __stack_chk_fail_local, but reportedly it appears prior to -lc in the link order, thus failing to satisfy references from libc itself (which arise only if libc.a was built as PIC/PIE with stack protector enabled).
2015-06-20work around mips detached thread exit breakage due to kernel regressionRich Felker1-0/+1
linux kernel commit 46e12c07b3b9603c60fc1d421ff18618241cb081 caused the mips syscall mechanism to fail with EFAULT when the userspace stack pointer is invalid, breaking __unmapself used for detached thread exit. the workaround is to set $sp to a known-valid, readable address, and the simplest one to obtain is the address of the current function, which is available (per o32 calling convention) in $25.
2015-06-17ignore ENOSYS error from mprotect in pthread_create and dynamic linkerRich Felker2-3/+6
this error simply indicated a system without memory protection (NOMMU) and should not cause failure in the caller.
2015-06-16switch to using trap number 31 for syscalls on shRich Felker7-10/+10
nominally the low bits of the trap number on sh are the number of syscall arguments, but they have never been used by the kernel, and some code making syscalls does not even know the number of arguments and needs to pass an arbitrary high number anyway. sh3/sh4 traditionally used the trap range 16-31 for syscalls, but part of this range overlapped with hardware exceptions/interrupts on sh2 hardware, so an incompatible range 32-47 was chosen for sh2. using trap number 31 everywhere, since it's in the existing sh3/sh4 range and does not conflict with sh2 hardware, is a proposed unification of the kernel syscall convention that will allow binaries to be shared between sh2 and sh3/sh4. if this is not accepted into the kernel, we can refit the sh2 target with runtime selection mechanisms for the trap number, but doing so would be invasive and would entail non-trivial overhead.
2015-06-16switch sh port's __unmapself to generic version when running on sh2/nommuRich Felker1-3/+3
due to the way the interrupt and syscall trap mechanism works, userspace on sh2 must never set the stack pointer to an invalid value. thus, the approach used on most archs, where __unmapself executes with no stack for the interval between SYS_munmap and SYS_exit, is not viable on sh2. in order not to pessimize sh3/sh4, the sh asm version of __unmapself is not removed. instead it's renamed and redirected through code that calls either the generic (safe) __unmapself or the sh3/sh4 asm, depending on compile-time and run-time conditions.
2015-06-16add support for sh2 interrupt-masking-based atomics to sh portRich Felker1-6/+0
the sh2 target is being considered an ISA subset of sh3/sh4, in the sense that binaries built for sh2 are intended to be usable on later cpu models/kernels with mmu support. so rather than hard-coding sh2-specific atomics, the runtime atomic selection mechanisms that was already in place has been extended to add sh2 atomics. at this time, the sh2 atomics are not SMP-compatible; since the ISA lacks actual atomic operations, the new code instead masks interrupts for the duration of the atomic operation, producing an atomic result on single-core. this is only possible because the kernel/hardware does not impose protections against userspace doing so. additional changes will be needed to support future SMP systems. care has been taken to avoid producing significant additional code size in the case where it's known at compile-time that the target is not sh2 and does not need sh2-specific code.
2015-06-16refactor stdio open file list handling, move it out of global libc structRich Felker12-41/+42
functions which open in-memory FILE stream variants all shared a tail with __fdopen, adding the FILE structure to stdio's open file list. replacing this common tail with a function call reduces code size and duplication of logic. the list is also partially encapsulated now. function signatures were chosen to facilitate tail call optimization and reduce the need for additional accessor functions. with these changes, static linked programs that do not use stdio no longer have an open file list at all.
2015-06-16byte-based C locale, phase 3: make MB_CUR_MAX variable to activate codeRich Felker1-0/+3
this patch activates the new byte-based C locale (high bytes treated as abstract code unit "characters" rather than decoded as multibyte characters) by making the value of MB_CUR_MAX depend on the active locale. for the C locale, the LC_CTYPE category pointer is null, yielding a value of 1. all other locales yield a value of 4.
2015-06-16byte-based C locale, phase 2: stdio and iconv (multibyte callers)Rich Felker7-8/+40
this patch adjusts libc components which use the multibyte functions internally, and which depend on them operating in a particular encoding, to make the appropriate locale changes before calling them and restore the calling thread's locale afterwards. activating the byte-based C locale without these changes would cause regressions in stdio and iconv. in the case of iconv, the current implementation was simply using the multibyte functions as UTF-8 conversions. setting a multibyte UTF-8 locale for the duration of the iconv operation allows the code to continue working. in the case of stdio, POSIX requires that FILE streams have an encoding rule bound at the time of setting wide orientation. as long as all locales, including the C locale, used the same encoding, treating high bytes as UTF-8, there was no need to store an encoding rule as part of the stream's state. a new locale field in the FILE structure points to the locale that should be made active during fgetwc/fputwc/ungetwc on the stream. it cannot point to the locale active at the time the stream becomes oriented, because this locale could be mutable (the global locale) or could be destroyed (locale_t objects produced by newlocale) before the stream is closed. instead, a pointer to the static C or C.UTF-8 locale object added in commit commit aeeac9ca5490d7d90fe061ab72da446c01ddf746 is used. this is valid since categories other than LC_CTYPE will not affect these functions.
2015-06-16byte-based C locale, phase 1: multibyte character handling functionsRich Felker10-7/+53
this patch makes the functions which work directly on multibyte characters treat the high bytes as individual abstract code units rather than as multibyte sequences when MB_CUR_MAX is 1. since MB_CUR_MAX is presently defined as a constant 4, all of the new code added is dead code, and optimizing compilers' code generation should not be affected at all. a future commit will activate the new code. as abstract code units, bytes 0x80 to 0xff are represented by wchar_t values 0xdf80 to 0xdfff, at the end of the surrogates range. this ensures that they will never be misinterpreted as Unicode characters, and that all wctype functions return false for these "characters" without needing locale-specific logic. a high range outside of Unicode such as 0x7fffff80 to 0x7fffffff was also considered, but since C11's char16_t also needs to be able to represent conversions of these bytes, the surrogate range was the natural choice.
2015-06-16fix btowc corner caseRich Felker1-0/+1
btowc is required to interpret its argument by conversion to unsigned char, unless the argument is equal to EOF. since the conversion to produces a non-character value anyway, we can just unconditionally convert, for now.
2015-06-14refactor malloc's expand_heap to share with __simple_mallocRich Felker3-81/+126
this extends the brk/stack collision protection added to full malloc in commit 276904c2f6bde3a31a24ebfa201482601d18b4f9 to also protect the __simple_malloc function used in static-linked programs that don't reference the free function. it also extends support for using mmap when brk fails, which full malloc got in commit 5446303328adf4b4e36d9fba21848e6feb55fab4, to __simple_malloc. since __simple_malloc may expand the heap by arbitrarily large increments, the stack collision detection is enhanced to detect interval overlap rather than just proximity of a single address to the stack. code size is increased a bit, but this is partly offset by the sharing of code between the two malloc implementations, which due to linking semantics, both get linked in a program that needs the full malloc with realloc/free support.
2015-06-13remove cancellation points in stdioRich Felker3-24/+3
commit 58165923890865a6ac042fafce13f440ee986fd9 added these optional cancellation points on the basis that cancellable stdio could be useful, to unblock threads stuck on stdio operations that will never complete. however, the only way to ensure that cancellation can achieve this is to violate the rules for side effects when cancellation is acted upon, discarding knowledge of any partial data transfer already completed. our implementation exhibited this behavior and was thus non-conforming. in addition to improving correctness, removing these cancellation points moderately reduces code size, and should significantly improve performance on i386, where sysenter/syscall instructions can be used instead of "int $128" for non-cancellable syscalls.
2015-06-13fix idiom for setting stdio stream orientation to wideRich Felker6-6/+6
the old idiom, f->mode |= f->mode+1, was adapted from the idiom for setting byte orientation, f->mode |= f->mode-1, but the adaptation was incorrect. unless the stream was alreasdy set byte-oriented, this code incremented f->mode each time it was executed, which would eventually lead to overflow. it could be fixed by changing it to f->mode |= 1, but upcoming changes will require slightly more work at the time of wide orientation, so it makes sense to just call fwide. as an optimization in the single-character functions, fwide is only called if the stream is not already wide-oriented.
2015-06-13add printing of null %s arguments as "(null)" in wide printfRich Felker1-0/+1
this is undefined, but supported in our implementation of the normal printf, so for consistency the wide variant should support it too.
2015-06-13add %m support to wide printfRich Felker1-0/+2
2015-06-11add sh asm for vforkRich Felker1-0/+23
2015-06-10implement arch-generic version of __unmapselfRich Felker1-0/+29
this can be used to put off writing an asm version of __unmapself for new archs, or as a permanent solution on archs where it's not practical or even possible to run momentarily with no stack. the concept here is simple: the caller takes a lock on a global shared stack and uses it to make the munmap and exit syscalls. the only trick is unlocking, which must be done after the thread exits, and this is achieved by using the set_tid_address syscall to have the kernel zero and futex-wake the lock word as part of the exit syscall.
2015-06-09in malloc, refuse to use brk if it grows into stackRich Felker1-1/+9
the linux/nommu fdpic ELF loader sets up the brk range to overlap entirely with the main thread's stack (but growing from opposite ends), so that the resulting failure mode for malloc is not to return a null pointer but to start returning pointers to memory that overlaps with the caller's stack. needless to say this extremely dangerous and makes brk unusable. since it's non-trivial to detect execution environments that might be affected by this kernel bug, and since the severity of the bug makes any sort of detection that might yield false-negatives unsafe, we instead check the proximity of the brk to the stack pointer each time the brk is to be expanded. both the main thread's stack (where the real known risk lies) and the calling thread's stack are checked. an arbitrary gap distance of 8 MB is imposed, chosen to be larger than linux default main-thread stack reservation sizes and larger than any reasonable stack configuration on nommu. the effeciveness of this patch relies on an assumption that the amount by which the brk is being grown is smaller than the gap limit, which is always true for malloc's use of brk. reliance on this assumption is why the check is being done in malloc-specific code and not in __brk.
2015-06-09fix spurious errors from pwd/grp functions when nscd backend is absentRich Felker1-4/+8
for several pwd/grp functions, the only way the caller can distinguish between a successful negative result ("no such user/group") and an internal error is by clearing errno before the call and checking errno afterwards. the nscd backend support code correctly simulated a not-found response on systems where such a backend is not running, but failed to restore errno. this commit also fixed an outdated/incorrect comment.
2015-06-07fix regression in pre-v7 arm on kernels with kuser helper removedRich Felker1-17/+14
the arm atomics/TLS runtime selection code is called from __set_thread_area and depends on having libc.auxv and __hwcap available. commit 71f099cb7db821c51d8f39dfac622c61e54d794c moved the first call to __set_thread_area to the top of dynamic linking stage 3, before this data is made available, causing the runtime detection code to always see __hwcap as zero and thereby select the atomics/TLS implementations based on kuser helper. upcoming work on superh will use similar runtime detection. ideally this early-init code should be cleanly refactored and shared between the dynamic linker and static-linked startup.
2015-06-07add multiple inclusion guard to locale_impl.hRich Felker1-0/+5
2015-06-07remove redefinition of MB_CUR_MAX in locale_impl.hRich Felker1-3/+0
unless/until the byte-based C locale is implemented, defining MB_CUR_MAX to 1 in the C locale is wrong. no internal code currently uses the MB_CUR_MAX macro, but having it defined inconsistently is error-prone. applications get the value from stdlib.h and were unaffected.
2015-06-06make static C and C.UTF-8 locales available outside of newlocaleRich Felker4-21/+28
2015-06-06remove another invalid skip of locking in ungetwcRich Felker1-3/+1
2015-06-06add macro version of ctype.h isascii functionRich Felker1-0/+1
presumably internal code (ungetwc and fputwc) was written assuming a macro implementation existed; otherwise use of isascii is just a pessimization.
2015-06-06remove invalid skip of locking in ungetwcRich Felker1-6/+3
aside from being invalid, the early check only optimized the error case, and likely pessimized the common case by separating the two branches on isascii(c) at opposite ends of the function.
2015-06-05fix uselocale((locale_t)0) not to modify localeTimo Teräs1-3/+1
commit 68630b55c0c7219fe9df70dc28ffbf9efc8021d8 made the new locale to be assigned unconditonally resulting in crashes later on.
2015-06-04fix dynamic linker regression processing R_*_NONE type relocationsRich Felker1-0/+1
commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 inadvertently removed the early check for "none" type relocations, causing the address dso->base+0 to be dereferenced to obtain an addend. shared libraries, (including libc.so) and PIE executables were unaffected, since their base addresses are the actual address of their mappings and are readable. non-PIE main executables, however, have a base address of 0 because their load addresses are absolute and not offset at load time. in practice none-type relocations do not arise with toolchains that are in use except on mips, and on mips it's moderately rare for a non-PIE executable to have a relocation table, since the mips-specific got processing serves in its place for most purposes.
2015-05-29fix failure of ungetc and ungetwc to work on files in eof statusRich Felker5-10/+11
these functions were written to handle clearing eof status, but failed to account for the __toread function's handling of eof. with this patch applied, __toread still returns EOF when the file is in eof status, so that read operations will fail, but it also sets up valid buffer pointers for read mode, which are set to the end of the buffer rather than the beginning in order to make the whole buffer available to ungetc/ungetwc. minor changes to __uflow were needed since it's now possible to have non-zero buffer pointers while in eof status. as made, these changes remove a 'fast path' bypassing the function call to __toread, which could be reintroduced with slightly different logic, but since ordinary files have a syscall in f->read, optimizing the code path does not seem worthwhile. the __stdio_read function is also updated not to zero the read buffer pointers on eof/error. while not necessary for correctness, this change avoids the overhead of calling __toread in ungetc after reaching eof, and it also reduces code size and increases consistency with the fmemopen read operation which does not zero the pointers.
2015-05-27implement fail-safe static locales for newlocaleRich Felker3-13/+46
this frees applications which need to make temporary use of the C locale (via uselocale) from the possibility that newlocale might fail. the C.UTF-8 locale is also provided as a static locale. presently they behave the same, but this may change in the future.
2015-05-27rename internal locale file handling locale mapsRich Felker1-0/+0
since the __setlocalecat function was removed, the filename __setlocalecat.c no longer made sense.
2015-05-27overhaul locale internals to treat categories roughly uniformlyRich Felker8-136/+112
previously, LC_MESSAGES was treated specially as the only category which could be set to a locale name without a definition file, in order to facilitate gettext message translations when no libc locale was available. LC_NUMERIC was completely un-settable, and LC_CTYPE stored a flag intended to be used for a possible future byte-based C locale, instead of storing a __locale_map pointer like the other categories use. this patch changes all categories to be represented by pointers to __locale_map structures, and allows locale names without definition files to be treated as valid locales with trivial definition when used in any category. outwardly visible functional changes should be minor, limited mainly to the strings read back from setlocale and the way gettext handles translations in categories other than LC_MESSAGES. various internal refactoring has also been performed, and improvements in const correctness have been made.
2015-05-27replace atomics with locks in locale-setting codeRich Felker2-32/+51
this is part of a general program of removing direct use of atomics where they are not necessary to meet correctness or performance needs, but in this case it's also an optimization. only the global locale needs synchronization; allocated locales referenced with locale_t handles are immutable during their lifetimes, and using atomics to initialize them increases their cost of setup.
2015-05-25reprocess all libc/ldso symbolic relocations in dynamic linking stage 3Rich Felker1-11/+43
commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 introduced early relocations and subsequent reprocessing as part of the dynamic linker bootstrap overhaul, to allow use of arbitrary libc functions before the main application and libraries are loaded, but only reprocessed GOT/PLT relocation types. commit c093e2e8201524db0d638920e76bcb6b1d925f3a added reprocessing of non-GOT/PLT relocations to fix an actual regression that was observed on powerpc, but only for RELA format tables with out-of-line addends. REL table (inline addends at the relocation address) reprocessing is trickier because the first relocation pass clobbers the addends. this patch extends symbolic relocation reprocessing for libc/ldso to support all relocation types, whether REL or RELA format tables are used. it is believed not to alter behavior on any existing archs for the current dynamic linker and libc code. the motivations for this change are consistency and future-proofing. it ensures that behavior does not differ depending on whether REL or RELA tables are used, which could lead to undetected arch-specific bugs. it also ensures that, if in the future code depending on additional relocation types is added to libc.so, either at the source level or as part of the compiler runtime that gets pulled in (for example, soft-float with TLS for fenv), the new code will work properly. the implementation concept is simple: stage 2 of the dynamic linker counts the number of symbolic relocations in the libc/ldso REL table and allocates a VLA to save their addends into; stage 3 then uses the saved addends in place of the inline ones which were clobbered. for stack safety, a hard limit (currently 4k) is imposed on the number of such addends; this should be a couple orders of magnitude larger than the actual need. this number is not a runtime variable that could break fail-safety; it is constant for a given libc.so build.
2015-05-25move call to dynamic linker stage-3 into stage-2 functionRich Felker3-12/+9
this move eliminates a duplicate "by-hand" symbol lookup loop from the stage-1 code and replaces it with a call to find_sym, which can be used once we're in stage 2. it reduces the size of the stage 1 code, which is helpful because stage 1 will become the crt start file for static-PIE executables, and it will allow stage 3 to access stage 2's automatic storage, which will be important in an upcoming commit.
2015-05-25mark mips cancellable syscall code as codeRich Felker1-0/+3
otherwise disassemblers treat it as data.
2015-05-25simplify/shrink relocation processing in dynamic linker stage 1Rich Felker1-24/+16
the outer-loop approach made sense when we were also processing DT_JMPREL, which might be in REL or RELA form, to avoid major code duplication. commit 09db855b35709aa627d7055c57a98e1e471920ab removed processing of DT_JMPREL, and in the remaining two tables, the format (REL or RELA) is known by the name of the table. simply writing two versions of the loop results in smaller and simpler code.
2015-05-25remove processing of DT_JMPREL from dynamic linker stage 1 bootstrapRich Felker1-1/+0
the DT_JMPREL relocation table necessarily consists entirely of JMP_SLOT (REL_PLT in internal nomenclature) relocations, which are symbolic; they cannot be resolved in stage 1, so there is no point in processing them.
2015-05-21remove outdated and misleading comment in iconv.cRich Felker1-6/+0
the comment claimed that EUC/GBK/Big5 are not implemented, which has been incorrect since commit 19b4a0a20efc6b9df98b6a43536ecdd628ba4643.
2015-05-21in iconv_open, accept "CHAR" and "" as aliases for "UTF-8"Rich Felker1-1/+2
while not a requirement, it's common convention in other iconv implementations to accept "CHAR" as an alias for nl_langinfo(CODESET), meaning the encoding used for char[] strings in the current locale, and also "" as an alternate form. supporting this is not costly and improves compatibility.
2015-05-18reprocess libc/ldso RELA relocations in stage 3 of dynamic linkingRich Felker1-1/+1
this fixes a regression on powerpc that was introduced in commit f3ddd173806fd5c60b3f034528ca24542aecc5b9. global data accesses on powerpc seem to be using a translation-unit-local GOT filled via R_PPC_ADDR32 relocations rather than R_PPC_GLOB_DAT. being a non-GOT relocation type, these were not reprocessed after adding the main application and its libraries to the chain, causing libc code not to see copy relocations in the main program, and therefore to use the pre-copy-relocation addresses for global data objects (like environ). the motivation for the dynamic linker only reprocessing GOT/PLT relocation types in stage 3 is that these types always have a zero addend, making them safe to process again even if the storage for the addend has been clobbered. other relocation types which can be used for address constants in initialized data objects may have non-zero addends which will be clobbered during the first pass of relocation processing if they're stored inline (REL form) rather than out-of-line (RELA form). powerpc generally uses only RELA, so this patch is sufficient to fix the regression in practice, but is not fully general, and would not suffice if an alternate toolchain generated REL for powerpc.