summaryrefslogtreecommitdiff
path: root/.github/workflows
diff options
context:
space:
mode:
authorPeter Scheibel <scheibel1@llnl.gov>2020-11-12 12:26:23 -0800
committerGitHub <noreply@github.com>2020-11-12 12:26:23 -0800
commitbb42470211a82fc333d48ba56c0d0990cd1faec5 (patch)
tree2bf23bb4ed77e2d9462cdb0b46561a0dd9563b95 /.github/workflows
parent81cab3b2711f12d82f73db44438afd5794529f1e (diff)
downloadspack-bb42470211a82fc333d48ba56c0d0990cd1faec5.tar.gz
spack-bb42470211a82fc333d48ba56c0d0990cd1faec5.tar.bz2
spack-bb42470211a82fc333d48ba56c0d0990cd1faec5.tar.xz
spack-bb42470211a82fc333d48ba56c0d0990cd1faec5.zip
macos: update build process to use spawn instead of fork (#18205)
Spack creates a separate process to do package installation. Different operating systems and Python versions use different methods to create it but up until Python 3.8 both Linux and Mac OS used "fork" (which duplicates process memory, file descriptor table, etc.). Python >= 3.8 on Mac OS prefers creating an entirely new process (referred to as the "spawn" start method) because "fork" was found to cause issues (in other words "spawn" is the default start method used by multiprocessing.Process). Spack was dependent on the particular behavior of fork to replicate process memory and transmit file descriptors. This PR refactors the Spack internals to support starting a child process with the "spawn" method. To achieve this, it makes the following changes: - ensure that the package repository and other global state are transmitted to the child process - ensure that file descriptors are transmitted to the child process in a way that works with multiprocessing and spawn - make all the state needed for the build process and tests picklable (package, stage, etc.) - move a number of locally-defined functions into global scope so that they can be pickled - rework tests where needed to avoid using local functions This PR also reworks sbang tests to work on macOS, where temporary directories are deeper than the Linux sbang limit. We make the limit platform-dependent (macOS supports 512-character shebangs) See: #14102
Diffstat (limited to '.github/workflows')
-rw-r--r--.github/workflows/macos_unit_tests.yaml5
1 files changed, 4 insertions, 1 deletions
diff --git a/.github/workflows/macos_unit_tests.yaml b/.github/workflows/macos_unit_tests.yaml
index d8e5e631b6..22baac3608 100644
--- a/.github/workflows/macos_unit_tests.yaml
+++ b/.github/workflows/macos_unit_tests.yaml
@@ -12,13 +12,16 @@ on:
jobs:
build:
runs-on: macos-latest
+ strategy:
+ matrix:
+ python-version: [3.8, 3.9]
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: actions/setup-python@v2
with:
- python-version: 3.9
+ python-version: ${{ matrix.python-version }}
- name: Install Python packages
run: |
pip install --upgrade pip six setuptools