summaryrefslogtreecommitdiff
path: root/bin
diff options
context:
space:
mode:
authorTom Scogland <scogland1@llnl.gov>2024-03-21 01:32:28 -0700
committerGitHub <noreply@github.com>2024-03-21 01:32:28 -0700
commit0eb1957999ccd9e54685f0e69c3a99df908c3e12 (patch)
treedde6125189c2f36d153611f0668c428e80cab4c8 /bin
parentde1f9593c618417b4a3fdd1f7544cb09609408a8 (diff)
downloadspack-0eb1957999ccd9e54685f0e69c3a99df908c3e12.tar.gz
spack-0eb1957999ccd9e54685f0e69c3a99df908c3e12.tar.bz2
spack-0eb1957999ccd9e54685f0e69c3a99df908c3e12.tar.xz
spack-0eb1957999ccd9e54685f0e69c3a99df908c3e12.zip
cmd/python: use runpy to allow multiprocessing in scripts (#41789)
Running a `spack-python` script like this: ```python import spack import multiprocessing def echo(args): print(args) if __name__ == "__main__": pool = multiprocessing.Pool(2) pool.map(echo, range(10)) ``` will fail in `develop` with an error like this: ```console _pickle.PicklingError: Can't pickle <function echo at 0x104865820>: attribute lookup echo on __main__ failed ``` Python expects to be able to look up the method `echo` in `sys.path["__main__"]` in subprocesses spawned by `multiprocessing`, but because we use `InteractiveConsole` to run `spack python`, the executed file isn't considered to be the `__main__` module, and lookups in subprocesses fail. We tried to fake this by setting `__name__` to `__main__` in the `spack python` command, but that doesn't fix the fact that no `__main__` module exists. Another annoyance with `InteractiveConsole` is that `__file__` is not defined in the main script scope, so you can't use it in your scripts. We can use the [runpy.run_path()](https://docs.python.org/3/library/runpy.html#runpy.run_path) function, which has been around since Python 3.2, to fix this. - [x] Use `runpy` module to launch non-interactive `spack python` invocations - [x] Only use `InteractiveConsole` for interactive `spack python`
Diffstat (limited to 'bin')
0 files changed, 0 insertions, 0 deletions