From bb42470211a82fc333d48ba56c0d0990cd1faec5 Mon Sep 17 00:00:00 2001 From: Peter Scheibel Date: Thu, 12 Nov 2020 12:26:23 -0800 Subject: macos: update build process to use spawn instead of fork (#18205) Spack creates a separate process to do package installation. Different operating systems and Python versions use different methods to create it but up until Python 3.8 both Linux and Mac OS used "fork" (which duplicates process memory, file descriptor table, etc.). Python >= 3.8 on Mac OS prefers creating an entirely new process (referred to as the "spawn" start method) because "fork" was found to cause issues (in other words "spawn" is the default start method used by multiprocessing.Process). Spack was dependent on the particular behavior of fork to replicate process memory and transmit file descriptors. This PR refactors the Spack internals to support starting a child process with the "spawn" method. To achieve this, it makes the following changes: - ensure that the package repository and other global state are transmitted to the child process - ensure that file descriptors are transmitted to the child process in a way that works with multiprocessing and spawn - make all the state needed for the build process and tests picklable (package, stage, etc.) - move a number of locally-defined functions into global scope so that they can be pickled - rework tests where needed to avoid using local functions This PR also reworks sbang tests to work on macOS, where temporary directories are deeper than the Linux sbang limit. We make the limit platform-dependent (macOS supports 512-character shebangs) See: #14102 --- bin/spack | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) (limited to 'bin') diff --git a/bin/spack b/bin/spack index 9d8e0419ad..7e2e7e06de 100755 --- a/bin/spack +++ b/bin/spack @@ -59,6 +59,8 @@ if 'ruamel.yaml' in sys.modules: if 'ruamel' in sys.modules: del sys.modules['ruamel'] -# Once we've set up the system path, run the spack main method import spack.main # noqa -sys.exit(spack.main.main()) + +# Once we've set up the system path, run the spack main method +if __name__ == "__main__": + sys.exit(spack.main.main()) -- cgit v1.2.3-70-g09d2