summaryrefslogtreecommitdiff
path: root/arch/sh
AgeCommit message (Collapse)AuthorFilesLines
2020-02-05remove legacy time32 timer[fd] syscalls from public syscall.hRich Felker1-4/+4
this extends commit 5a105f19b5aae79dd302899e634b6b18b3dcd0d6, removing timer[fd]_settime and timer[fd]_gettime. the timerfd ones are likely to have been used in software that started using them before it could rely on libc exposing functions.
2020-02-05remove further legacy time32 clock syscalls from public syscall.hRich Felker1-4/+4
this extends commit 5a105f19b5aae79dd302899e634b6b18b3dcd0d6, removing clock_settime, clock_getres, clock_nanosleep, and settimeofday.
2020-01-30remove legacy clock_gettime and gettimeofday from public syscall.hRich Felker1-2/+2
some nontrivial number of applications have historically performed direct syscalls for these operations rather than using the public functions. such usage is invalid now that time_t is 64-bit and these syscalls no longer match the types they are used with, and it was already harmful before (by suppressing use of vdso). since syscall() has no type safety, incorrect usage of these syscalls can't be caught at compile-time. so, without manually inspecting or running additional tools to check sources, the risk of such errors slipping through is high. this patch renames the syscalls on 32-bit archs to clock_gettime32 and gettimeofday_time32, so that applications using the original names will fail to build without being fixed. note that there are a number of other syscalls that may also be unsafe to use directly after the time64 switchover, but (1) these are the main two that seem to be in widespread use, and (2) most of the others continue to have valid usage with a null timeval/timespec argument, as the argument is an optional timeout or similar.
2019-12-30add pidfd_open syscall number from linux v5.3Szabolcs Nagy1-0/+1
see linux commit 7615d9e1780e26e0178c93c55b73309a5dc093d7 arch: wire-up pidfd_open() linux commit 32fcb426ec001cb6d5a4a195091a8486ea77e2df pid: add pidfd_open()
2019-11-02move time_t and suseconds_t definitions to common alltypes.h.inRich Felker1-3/+0
now that all 32-bit archs have 64-bit time_t (and suseconds_t), the arch-provided _Int64 macro (long or long long, as appropriate) can be used to define them, and arch-specific definitions are no longer needed.
2019-11-02move time64 socket options from arch bits to top-level sys/socket.hRich Felker1-5/+0
now that all 32-bit archs have 64-bit time types, the values for the time-related socket option macros can be treated as universal for 32-bit archs. the sys/socket.h mechanism for this predates arch/generic and is instead in the top-level header. x32, which does not use the new time64 values of the macros, already has its own overrides, so this commit does not affect it.
2019-11-02switch all existing 32-bit archs to 64-bit time_tRich Felker9-22/+42
this commit preserves ABI fully for existing interface boundaries between libc and libc consumers (applications or libraries), by retaining existing symbol names for the legacy 32-bit interfaces and redirecting sources compiled against the new headers to alternate symbol names. this does not necessarily, however, preserve the pairwise ABI of libc consumers with one another; where they use time_t-derived types in their interfaces with one another, it may be necessary to synchronize updates with each other. the intent is that ABI resulting from this commit already be stable and permanent, but it will not be officially so until a release is made. changes to some header-defined types that do not play any role in the ABI between libc and its consumers may still be subject to change. mechanically, the changes made by this commit for each 32-bit arch are as follows: - _REDIR_TIME64 is defined to activate the symbol redirections in public headers - COMPAT_SRC_DIRS is defined in arch.mak to activate build of ABI compat shims to serve as definitions for the original symbol names - time_t and suseconds_t definitions are changed to long long (64-bit) - IPC_STAT definition is changed to add the IPC_TIME64 bit (0x100), triggering conversion of semid_ds, shmid_ds, and msqid_ds split low/high time bits into new time_t members - structs semid_ds, shmid_ds, msqid_ds, and stat are modified to add new 64-bit time_t/timespec members at the end, maintaining existing layout of other members. - socket options (SO_*) and ioctl (sockios) command macros are redefined to use the kernel's "_NEW" values. in addition, on archs where vdso clock_gettime is used, the VDSO_CGT_SYM macro definition in syscall_arch.h is changed to use a new time64 vdso function if available, and a new VDSO_CGT32_SYM macro is added for use as fallback on kernels lacking time64.
2019-10-17move pthread types out of per-arch alltypes.hRich Felker1-8/+0
policy has long been that these definitions are purely a function of whether long/pointer is 32- or 64-bit, and that they are not allowed to vary per-arch. move the definition to the shared alltypes.h.in fragment, using integer constant expressions in terms of sizeof to vary the array dimensions appropriately. I'm not sure whether this is more or less ugly than using preprocessor conditionals and two sets of definitions here, but either way is a lot less ugly than repeating the same thing for every arch.
2019-10-17define LONG_MAX via arch alltypes.h, strip down bits/limits.hRich Felker2-7/+2
LLONG_MAX is uniform for all archs we support and plenty of header and code level logic assumes it is, so it does not make sense for limits.h bits mechanism to pretend it's variable. LONG_BIT can be defined in terms of LONG_MAX; there's no reason to put it in bits. by moving LONG_MAX definition to __LONG_MAX in alltypes.h and moving LLONG_MAX out of bits, there are now no plain-C limits that are defined in the bits header, so the bits header only needs to be included in the POSIX or extended profiles. this allows the feature test macro logic to be removed from the bits header, facilitating a long-term goal of getting such logic out of bits. having __LONG_MAX in alltypes.h will allow further generalization of headers. archs without a constant PAGESIZE no longer need bits/limits.h at all.
2019-10-17remove use of endian.h from arch reloc.h headers, clean upRich Felker1-2/+0
building on commit 97d35a552ec5b6ddf7923dd2f9a8eb973526acea, __BYTE_ORDER is now available wherever alltypes.h is included. since reloc.h is only used from src/internal/dynlink.h, it can be assumed that __BYTE_ORDER is exposed. reloc.h is not permitted to be included in other contexts, and generally, like most arch headers, lacks inclusion guards that would allow such usage. the mips64 version mistakenly included such guards; they are removed for consistency.
2019-10-17move __BYTE_ORDER definition to alltypes.hRich Felker2-5/+6
this change is motivated by the intersection of several factors. presently, despite being a nonstandard header, endian.h is exposing the unprefixed byte order macros and functions only if _BSD_SOURCE or _GNU_SOURCE is defined. this is to accommodate use of endian.h from other headers, including bits headers, which need to define structure layout in terms of endianness. with time64 switch-over, even more headers will need to do this. at the same time, the resolution of Austin Group issue 162 makes endian.h a standard header for POSIX-future, requiring that it expose the unprefixed macros and the functions even in standards-conforming profiles. changes to meet this new requirement would break existing internal usage of endian.h by causing it to violate namespace where it's used. instead, have the arch's alltypes.h define __BYTE_ORDER, either as a fixed constant or depending on the right arch-specific predefined macros for determining endianness. explicit literals 1234 and 4321 are used instead of __LITTLE_ENDIAN and __BIG_ENDIAN so that there's no danger of getting the wrong result if a macro is undefined and implicitly evaluates to 0 at the preprocessor level. the powerpc (32-bit) bits/endian.h being removed had logic for varying endianness, but our powerpc arch has never supported that and has always been big-endian-only. this logic is not carried over to the new __BYTE_ORDER definition in alltypes.h.
2019-10-17remove per-arch definitions for va_listRich Felker1-3/+0
now that commit f7f1079796abc6f97c69521d2334e9c7d3945dd8 removed the legacy i386 conditional definition, va_list is in no way arch-specific, and has no reason to be in the future. move it to the shared part of alltypes.h.in
2019-09-11add new syscall numbers from linux v5.2Szabolcs Nagy1-0/+6
new mount api syscalls were added, same numers on all targets, see linux commit a07b20004793d8926f78d63eb5980559f7813404 vfs: syscall: Add open_tree(2) to reference or clone a mount linux commit 2db154b3ea8e14b04fee23e3fdfd5e9d17fbc6ae vfs: syscall: Add move_mount(2) to move mounts around linux commit 24dcb3d90a1f67fe08c68a004af37df059d74005 vfs: syscall: Add fsopen() to prepare for superblock creation linux commit ecdab150fddb42fe6a739335257949220033b782 vfs: syscall: Add fsconfig() for configuring and managing a context linux commit 93766fbd2696c2c4453dd8e1070977e9cd4e6b6d vfs: syscall: Add fsmount() to create a mount for a superblock linux commit cf3cba4a429be43e5527a3f78859b1bfd9ebc5fb vfs: syscall: Add fspick() to select a superblock for reconfiguration linux commit 9c8ad7a2ff0bfe58f019ec0abc1fb965114dde7d uapi, x86: Fix the syscall numbering of the mount API syscalls [ver #2] linux commit d8076bdb56af5e5918376cd1573a6b0007fc1a89 uapi: Wire up the mount API syscalls on non-x86 arches [ver #2]
2019-09-08honor __WCHAR_TYPE__ on archs with legacy long definition of wchar_tRich Felker1-0/+4
historically, a number of 32-bit archs used long rather than int for wchar_t, for no good reason. GCC still uses the historical types, but clang replaced them all with int, and it seems PCC uses int too. mismatching the compiler's type for wchar_t is not an option due to wide string literals. note that the mismatch does not affect C++ ABI since wchar_t is its own builtin type/keyword in C++, distinct from both int and long, not a typedef. i386 already worked around this by honoring __WCHAR_TYPE__ if defined by the compiler, and only using the official legacy ABI type if not. add the same to the other affected archs. it might make sense at some point to switch to using int as the default if __WCHAR_TYPE__ is not defined, if the expectations is that new compilers will treat int as the correct choice, but it's unlikely that the case where __WCHAR_TYPE__ is undefined will ever be used anyway. I actually wanted to move the definition of wchar_t to the top-level shared alltypes.h.in, using __WCHAR_TYPE__ and falling back to int if not defined, but that can't be done without assuming all compilers define __WCHAR_TYPE__ thanks to some pathological archs where the ABI has wchar_t as an unsigned type.
2019-08-19fix clash between sys/user.h and kernel ptrace.h on powerpc[64], shRich Felker1-29/+5
due to historical accident/sloppiness in glibc, the powerpc, powerpc64, and sh versions of struct user, defined by sys/user.h, used struct pt_regs from the kernel asm/ptrace.h for their regs member. this made it impossible to define the type in an API-compatible manner without either including asm/ptrace.h like glibc does (contrary to our policy of not depending on kernel headers), or clashing with asm/ptrace.h's definition of struct pt_regs if both headers are included (which is almost always the case in software using sys/user.h). for a long time I viewed this problem as having no reasonable fix. I even explored the possibility of having the powerpc[64] and sh versions of user.h just include the kernel header (breaking with policy), but that looked like it might introduce new clashes with sys/ptrace.h. and it would also bring in a lot of additional cruft that makes no sense for sys/user.h to expose. glibc goes out of its way to suppress some of that with #undef, possibly leading to different problems. this is a rabbit-hole that should be explored no further. as it turns out, however, nothing actually uses struct user sufficiently to care about the type of the regs member; most software including sys/user.h does not even use struct user at all. so, the problem can be fixed just by doing away with the insistence on strict glibc API compatibility for the struct tag of the regs member. rather than renaming the tag, which might lead to the new name entering use as API, simply use an untagged structure inside struct user with the same members/layout as struct pt_regs. for sh, struct pt_dspregs is just removed entirely since it was not used.
2019-08-02move IPC_STAT definition to a new bits/ipcstat.h fileRich Felker1-0/+1
otherwise, 32-bit archs that could otherwise share the generic bits/ipc.h would need to duplicate the struct ipc_perm definition, obscuring the fact that it's the same. sysvipc is not widely used and these headers are not commonly included, so there is no performance gain to be had by limiting the number of indirectly included files here. files with the existing time32 definition of IPC_STAT are added to all current 32-bit archs now, so that when it's changed the change will show up as a change rather than addition of a new file where it's less obvious that the value is changing vs the generic one that was used before.
2019-07-31ioctl: add fallback for new time64 SIOCGSTAMP[NS]Rich Felker1-0/+3
without this, the SIOCGSTAMP and SIOCGSTAMPNS ioctl commands, for obtaining timestamps, would stop working on pre-5.1 kernels after time_t is switched to 64-bit and their values are changed to the new time64 versions. new code is written such that it's statically unreachable on 64-bit archs, and on existing 32-bit archs until the macro values are changed to activate 64-bit time_t.
2019-07-29duplicate generic bits/msg.h for each arch using it, in prep to changeRich Felker1-0/+15
2019-07-29duplicate generic bits/sem.h for each arch using it, in prep to changeRich Felker1-0/+16
2019-07-18decouple struct stat from kernel typeRich Felker1-0/+21
presently, all archs/ABIs have struct stat matching the kernel stat[64] type, except mips/mipsn32/mips64 which do conversion hacks in syscall_arch.h to work around bugs in the kernel type. this patch completely decouples them and adds a translation step to the success path of fstatat. at present, this is just a gratuitous copying, but it opens up multiple possibilities for future support for 64-bit time_t on 32-bit archs and for cleaned-up/unified ABIs. for clarity, the mips hacks are not yet removed in this commit, so the mips kstat structs still correspond to the output of the hacks in their syscall_arch.h files, not the raw kernel type. a subsequent commit will fix this.
2019-07-01add new syscall numbers from linux v5.1Szabolcs Nagy1-0/+39
syscall numbers are now synced up across targets (starting from 403 the numbers are the same on all targets other than an arch specific offset) IPC syscalls sem*, shm*, msg* got added where they were missing (except for semop: only semtimedop got added), the new semctl, shmctl, msgctl imply IPC_64, see linux commit 0d6040d4681735dfc47565de288525de405a5c99 arch: add split IPC system calls where needed new 64bit time_t syscall variants got added on 32bit targets, see linux commit 48166e6ea47d23984f0b481ca199250e1ce0730a y2038: add 64-bit time_t syscalls to all 32-bit architectures new async io syscalls got added, see linux commit 2b188cc1bb857a9d4701ae59aa7768b5124e262e Add io_uring IO interface linux commit edafccee56ff31678a091ddb7219aba9b28bc3cb io_uring: add support for pre-mapped user IO buffers a new syscall got added that uses the fd of /proc/<pid> as a stable handle for processes: allows sending signals without pid reuse issues, intended to eventually replace rt_sigqueueinfo, kill, tgkill and rt_tgsigqueueinfo, see linux commit 3eb39f47934f9d5a3027fe00d906a45fe3a15fad signal: add pidfd_send_signal() syscall on some targets (arm, m68k, s390x, sh) some previously missing syscall numbers got added as well.
2019-02-07move arch-invariant definitions out of bits/ioctl.hBobby Bingham1-96/+0
2018-10-16make thread-pointer-loading asm non-volatileRich Felker1-1/+1
this will allow the compiler to cache and reuse the result, meaning we no longer have to take care not to load it more than once for the sake of archs where the load may be expensive. depends on commit 1c84c99913bf1cd47b866ed31e665848a0da84a2 for correctness, since otherwise the compiler could hoist loads during stage 3 of dynamic linking before the initial thread-pointer setup.
2018-09-20add arm and sh bits/ptrace.hSzabolcs Nagy1-0/+5
These should have been added in commit df6d9450ea19fd71e52cf5cdb4c85beb73066394 that added target specific PTRACE_ macros, but somehow got missed.
2018-09-12apply hidden visibility to sigreturn code fragmentsRich Felker1-1/+3
these were overlooked in the declarations overhaul work because they are not properly declared, and the current framework even allows their declared types to vary by arch. at some point this should be cleaned up, but I'm not sure what the right way would be.
2018-09-05define and use internal macros for hidden visibility, weak refsRich Felker1-1/+3
this cleans up what had become widespread direct inline use of "GNU C" style attributes directly in the source, and lowers the barrier to increased use of hidden visibility, which will be useful to recovering some of the efficiency lost when the protected visibility hack was dropped in commit dc2f368e565c37728b0d620380b849c3a1ddd78f, especially on archs where the PLT ABI is costly.
2018-08-29fix async thread cancellation on sh-fdpicRich Felker1-0/+5
if __cp_cancel was reached via __syscall_cp, r12 will necessarily still contain a GOT pointer (for libc.so or for the static-linked main program) valid for entering __cancel. however, in the case of async cancellation, r12 may contain any scratch value; it's not necessarily even a valid GOT pointer for the code that was interrupted. unlike in commit 0ec49dab6794166d67fae4764ce7fdea42ea6103 where the corresponding issue was fixed for powerpc64, there is fundamentally no way for fdpic code to recompute its GOT pointer. so a new mechanism is introduced for cancel_handler to write a GOT register value into the interrupted context on archs where it is needed.
2018-06-20work around broken kernel struct ipc_perm on some big endian archsRich Felker1-0/+2
the mode member of struct ipc_perm is specified by POSIX to have type mode_t, which is uniformly defined as unsigned int. however, Linux defines it with type __kernel_mode_t, and defines __kernel_mode_t as unsigned short on some archs. since there is a subsequent padding field, treating it as a 32-bit unsigned int works on little endian archs, but the order is backwards on big endian archs with the erroneous definition. since multiple archs are affected, remedy the situation with fixup code in the affected functions (shmctl, semctl, and msgctl) rather than repeating the same shims in syscall_arch.h for every affected arch.
2018-06-02fix TLS layout of TLS variant I when there is a gap above TPSzabolcs Nagy2-3/+4
In TLS variant I the TLS is above TP (or above a fixed offset from TP) but on some targets there is a reserved gap above TP before TLS starts. This matters for the local-exec tls access model when the offsets of TLS variables from the TP are hard coded by the linker into the executable, so the libc must compute these offsets the same way as the linker. The tls offset of the main module has to be alignup(GAP_ABOVE_TP, main_tls_align). If there is no TLS in the main module then the gap can be ignored since musl does not use it and the tls access models of shared libraries are not affected. The previous setup only worked if (tls_align & -GAP_ABOVE_TP) == 0 (i.e. TLS did not require large alignment) because the gap was treated as a fixed offset from TP. Now the TP points at the end of the pthread struct (which is aligned) and there is a gap above it (which may also need alignment). The fix required changing TP_ADJ and __pthread_self on affected targets (aarch64, arm and sh) and in the tlsdesc asm the offset to access the dtv changed too.
2018-03-10reverse definition dependency between PAGESIZE and PAGE_SIZERich Felker1-1/+1
PAGESIZE is actually the version defined in POSIX base, with PAGE_SIZE being in the XSI option. use PAGESIZE as the underlying definition to facilitate making exposure of PAGE_SIZE conditional.
2017-11-05ioctl TIOCGPTPEER from linux v4.13Szabolcs Nagy1-0/+1
added for safe opening of peer end of pty in a mount namespace. new in linux commit c6325179238f1d4683edbec53d8322575d76d7e2
2017-08-29add SIOCGSTAMPNS socket ioctl macro to ioctl.hSzabolcs Nagy1-0/+1
it is defined in linux asm/sockios.h since commit ae40eb1ef30ab4120bd3c8b7e3da99ee53d27a23 (linux v2.6.22) but was missing from musl by accident. in musl the sockios macros are exposed in sys/ioctl.h together with other ioctl requests instead of in sys/socket.h because of namespace rules. (glibc has them in sys/socket.h under _GNU_SOURCE.)
2017-08-11fix build failure for sh4a due to missing colon in asm statementThomas Petazzoni1-1/+1
Due to a missing ":" in an asm() statement, the "memory" clobber is considered by gcc as an input operand and not a clobber, which causes a build failure.
2016-10-20add bits/hwcap.h and include it in sys/auxv.hSzabolcs Nagy1-0/+11
aarch64, arm, mips, mips64, mipsn32, powerpc, powerpc64 and sh have cpu feature bits defined in linux for AT_HWCAP auxv entry, so expose those in sys/auxv.h it seems the mips hwcaps were never exposed to userspace neither by linux nor by glibc, but that's most likely an oversight.
2016-10-20add sh syscall numbers from linux v4.8Szabolcs Nagy1-0/+14
sh was updated in linux commit 74bdaa611fa69368fb4032ad437af073d31116bd to have numbers for new syscalls.
2016-08-11fix pread/pwrite syscall calling convention on shRich Felker1-0/+1
despite sh not generally using register-pair alignment for 64-bit syscall arguments, there are arch-specific versions of the syscall entry points for pread and pwrite which include a dummy argument for alignment before the 64-bit offset argument.
2016-07-03make brace placement in public header struct definitions consistentRich Felker2-4/+2
placing the opening brace on the same line as the struct keyword/tag is the style I prefer and seems to be the prevailing practice in more recent additions. these changes were generated by the command: find include/ arch/*/bits -name '*.h' \ -exec sed -i '/^struct [^;{]*$/{N;s/\n/ /;}' {} + and subsequently checked by hand to ensure that the regex did not pick up any false positives.
2016-07-03remove termios2 related ioctls from sh ioctl.hSzabolcs Nagy1-4/+0
musl does not define these on other targets either.
2016-07-03add missing TIOC* macros to ioctl.hSzabolcs Nagy1-0/+2
these are defined in linux asm/ioctls.h. (powerpc64 and powerpc bits/ioctl.h are now identical)
2016-07-03add missing SIOCSIFNAME from linux/sockios.h to ioctl.hSzabolcs Nagy1-0/+1
glibc ioctl.h has it too.
2016-07-03remove ioctl macros that were removed from linux uapiSzabolcs Nagy1-2/+0
TIOCTTYGSTRUCT, TIOCGHAYESESP, TIOCSHAYESESP and TIOCM_MODEM_BITS were removed from the linux uapi and not present in glibc ioctl.h
2016-05-12deduplicate __NR_* and SYS_* syscall number definitionsBobby Bingham2-684/+341
2016-03-18deduplicate bits/mman.hSzabolcs Nagy1-60/+0
currently five targets use the same mman.h constants and the rest share most constants too, so move them to sys/mman.h before the bits/mman.h include where the differences can be corrected by redefinition of the macros. this fixes two minor bugs: POSIX_MADV_DONTNEED was wrong on most targets (it should be the same as MADV_DONTNEED), and sh defined the x86-only MAP_32BIT mmap flag.
2016-01-27deduplicate the bulk of the arch bits headersRich Felker12-408/+0
all bits headers that were identical for a number of 'clean' archs are moved to the new arch/generic tree. in addition, a few headers that differed only cosmetically from the new generic version are removed. additional deduplication may be possible in mman.h and in several headers (limits.h, posix.h, stdint.h) that mostly depend on whether the arch is 32- or 64-bit, but they are left alone for now because greater gains are likely possible with more invasive changes to header logic, which is beyond the scope of this commit.
2016-01-26add MCL_ONFAULT and MLOCK_ONFAULT mlockall and mlock2 flagsSzabolcs Nagy1-0/+1
they lock faulted pages into memory (useful when a small part of a large mapped file needs efficient access), new in linux v4.4, commit b0f205c2a3082dd9081f9a94e50658c5fa906ff1 MLOCK_* is not in the POSIX reserved namespace for sys/mman.h
2016-01-22remove sh port's __fpscr_values source fileRich Felker1-5/+0
commit f3ddd173806fd5c60b3f034528ca24542aecc5b9, the dynamic linker bootstrap overhaul, silently disabled the definition of __fpscr_values in this file since libc.so's copy of __fpscr_values now comes from crt_arch.h, the same place the public definition in the main program's crt1.o ultimately comes from. remove this file which is no longer in use.
2016-01-22move sh port's __shcall internal function from arch/sh/src to src treeRich Felker1-5/+0
2016-01-22move sh __unmapself code from arch/sh/src to main src treeRich Felker1-24/+0
2016-01-21overhaul sh atomics for new atomics framework, add j-core cas.l backendRich Felker4-287/+30
sh needs runtime-selected atomic backends since there are a number of supported models that use non-forwards-compatible (non-smp-compatible) atomic mechanisms. previously, the code paths for this were highly inefficient since they involved C function calls with multiple branches in the callee and heavy spills in the caller. the new code performs calls the runtime-selected asm fragment from inline asm with extremely minimal clobbers, rather than using a function call. for the sh4a case where the atomic mechanism is known and there is no forward-compatibility issue, the movli.l and movco.l instructions are provided as a_ll and a_sc, allowing the new shared atomic.h to generate efficient inline versions of all the basic atomic operations without needing a cas loop.
2016-01-21refactor internal atomic.hRich Felker1-72/+0
rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.