diff options
| author | Harmen Stoppels <me@harmenstoppels.nl> | 2024-05-06 16:17:35 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-05-06 16:17:35 +0200 |
| commit | 125206d44d6ecf167624531dff3631b126791849 (patch) | |
| tree | 411d04b4f0a64ca4bdc808f9f71bbc45f0db92f1 | |
| parent | a081b875b436022515ef2a284313ca887bb28bc4 (diff) | |
| download | spack-125206d44d6ecf167624531dff3631b126791849.tar.gz spack-125206d44d6ecf167624531dff3631b126791849.tar.bz2 spack-125206d44d6ecf167624531dff3631b126791849.tar.xz spack-125206d44d6ecf167624531dff3631b126791849.zip | |
python: always use a venv (#40773)
This commit adds a layer of indirection to improve build isolation with
and without external Python, as well as usability of environment views.
It adds `python-venv` as a dependency to all packages that `extends("python")`,
which has the following advantages:
1. Build isolation: only `PYTHONPATH` is considered in builds, not
user / system packages
2. Stable install layout: fixes the problem on Debian, RHEL and Fedora where
external / system python produces `bin/local` subdirs in Spack install prefixes.
3. Environment views are Python virtual environments (and if you add
`py-pip` things like `pip list` work)
Views work whether they're symlink, hardlink or copy type.
This commit additionally makes `spec["python"].command` return
`spec["python-venv"].command`. The rationale is that packages in repos we do
not own do not pass the underlying python to the build system, which could still
result in incorrectly computed install layouts.
Other attributes like `libs`, `headers` should be on `python` anyways and need no change.
55 files changed, 430 insertions, 497 deletions
diff --git a/lib/spack/docs/basic_usage.rst b/lib/spack/docs/basic_usage.rst index f26c6e5683..e49ca3073e 100644 --- a/lib/spack/docs/basic_usage.rst +++ b/lib/spack/docs/basic_usage.rst @@ -865,7 +865,7 @@ There are several different ways to use Spack packages once you have installed them. As you've seen, spack packages are installed into long paths with hashes, and you need a way to get them into your path. The easiest way is to use :ref:`spack load <cmd-spack-load>`, which is -described in the next section. +described in this section. Some more advanced ways to use Spack packages include: @@ -959,7 +959,86 @@ use ``spack find --loaded``. You can also use ``spack load --list`` to get the same output, but it does not have the full set of query options that ``spack find`` offers. -We'll learn more about Spack's spec syntax in the next section. +We'll learn more about Spack's spec syntax in :ref:`a later section <sec-specs>`. + + +.. _extensions: + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Python packages and virtual environments +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Spack can install a large number of Python packages. Their names are +typically prefixed with ``py-``. Installing and using them is no +different from any other package: + +.. code-block:: console + + $ spack install py-numpy + $ spack load py-numpy + $ python3 + >>> import numpy + +The ``spack load`` command sets the ``PATH`` variable so that the right Python +executable is used, and makes sure that ``numpy`` and its dependencies can be +located in the ``PYTHONPATH``. + +Spack is different from other Python package managers in that it installs +every package into its *own* prefix. This is in contrast to ``pip``, which +installs all packages into the same prefix, be it in a virtual environment +or not. + +For many users, **virtual environments** are more convenient than repeated +``spack load`` commands, particularly when working with multiple Python +packages. Fortunately Spack supports environments itself, which together +with a view are no different from Python virtual environments. + +The recommended way of working with Python extensions such as ``py-numpy`` +is through :ref:`Environments <environments>`. The following example creates +a Spack environment with ``numpy`` in the current working directory. It also +puts a filesystem view in ``./view``, which is a more traditional combined +prefix for all packages in the environment. + +.. code-block:: console + + $ spack env create --with-view view --dir . + $ spack -e . add py-numpy + $ spack -e . concretize + $ spack -e . install + +Now you can activate the environment and start using the packages: + +.. code-block:: console + + $ spack env activate . + $ python3 + >>> import numpy + +The environment view is also a virtual environment, which is useful if you are +sharing the environment with others who are unfamiliar with Spack. They can +either use the Python executable directly: + +.. code-block:: console + + $ ./view/bin/python3 + >>> import numpy + +or use the activation script: + +.. code-block:: console + + $ source ./view/bin/activate + $ python3 + >>> import numpy + +In general, there should not be much difference between ``spack env activate`` +and using the virtual environment. The main advantage of ``spack env activate`` +is that it knows about more packages than just Python packages, and it may set +additional runtime variables that are not covered by the virtual environment +activation script. + +See :ref:`environments` for a more in-depth description of Spack +environments and customizations to views. .. _sec-specs: @@ -1705,165 +1784,6 @@ check only local packages (as opposed to those used transparently from ``upstream`` spack instances) and the ``-j,--json`` option to output machine-readable json data for any errors. - -.. _extensions: - ---------------------------- -Extensions & Python support ---------------------------- - -Spack's installation model assumes that each package will live in its -own install prefix. However, certain packages are typically installed -*within* the directory hierarchy of other packages. For example, -`Python <https://www.python.org>`_ packages are typically installed in the -``$prefix/lib/python-2.7/site-packages`` directory. - -In Spack, installation prefixes are immutable, so this type of installation -is not directly supported. However, it is possible to create views that -allow you to merge install prefixes of multiple packages into a single new prefix. -Views are a convenient way to get a more traditional filesystem structure. -Using *extensions*, you can ensure that Python packages always share the -same prefix in the view as Python itself. Suppose you have -Python installed like so: - -.. code-block:: console - - $ spack find python - ==> 1 installed packages. - -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- - python@2.7.8 - -.. _cmd-spack-extensions: - -^^^^^^^^^^^^^^^^^^^^ -``spack extensions`` -^^^^^^^^^^^^^^^^^^^^ - -You can find extensions for your Python installation like this: - -.. code-block:: console - - $ spack extensions python - ==> python@2.7.8%gcc@4.4.7 arch=linux-debian7-x86_64-703c7a96 - ==> 36 extensions: - geos py-ipython py-pexpect py-pyside py-sip - py-basemap py-libxml2 py-pil py-pytz py-six - py-biopython py-mako py-pmw py-rpy2 py-sympy - py-cython py-matplotlib py-pychecker py-scientificpython py-virtualenv - py-dateutil py-mpi4py py-pygments py-scikit-learn - py-epydoc py-mx py-pylint py-scipy - py-gnuplot py-nose py-pyparsing py-setuptools - py-h5py py-numpy py-pyqt py-shiboken - - ==> 12 installed: - -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- - py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2 - py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10 - py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1 - py-matplotlib@1.4.2 py-pyparsing@2.0.3 py-six@1.9.0 - -The extensions are a subset of what's returned by ``spack list``, and -they are packages like any other. They are installed into their own -prefixes, and you can see this with ``spack find --paths``: - -.. code-block:: console - - $ spack find --paths py-numpy - ==> 1 installed packages. - -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- - py-numpy@1.9.1 ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/py-numpy@1.9.1-66733244 - -However, even though this package is installed, you cannot use it -directly when you run ``python``: - -.. code-block:: console - - $ spack load python - $ python - Python 2.7.8 (default, Feb 17 2015, 01:35:25) - [GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2 - Type "help", "copyright", "credits" or "license" for more information. - >>> import numpy - Traceback (most recent call last): - File "<stdin>", line 1, in <module> - ImportError: No module named numpy - >>> - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Using Extensions in Environments -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The recommended way of working with extensions such as ``py-numpy`` -above is through :ref:`Environments <environments>`. For example, -the following creates an environment in the current working directory -with a filesystem view in the ``./view`` directory: - -.. code-block:: console - - $ spack env create --with-view view --dir . - $ spack -e . add py-numpy - $ spack -e . concretize - $ spack -e . install - -We recommend environments for two reasons. Firstly, environments -can be activated (requires :ref:`shell-support`): - -.. code-block:: console - - $ spack env activate . - -which sets all the right environment variables such as ``PATH`` and -``PYTHONPATH``. This ensures that - -.. code-block:: console - - $ python - >>> import numpy - -works. Secondly, even without shell support, the view ensures -that Python can locate its extensions: - -.. code-block:: console - - $ ./view/bin/python - >>> import numpy - -See :ref:`environments` for a more in-depth description of Spack -environments and customizations to views. - -^^^^^^^^^^^^^^^^^^^^ -Using ``spack load`` -^^^^^^^^^^^^^^^^^^^^ - -A more traditional way of using Spack and extensions is ``spack load`` -(requires :ref:`shell-support`). This will add the extension to ``PYTHONPATH`` -in your current shell, and Python itself will be available in the ``PATH``: - -.. code-block:: console - - $ spack load py-numpy - $ python - >>> import numpy - -The loaded packages can be checked using ``spack find --loaded`` - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Loading Extensions via Modules -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Apart from ``spack env activate`` and ``spack load``, you can load numpy -through your environment modules (using ``environment-modules`` or -``lmod``). This will also add the extension to the ``PYTHONPATH`` in -your current shell. - -.. code-block:: console - - $ module load <name of numpy module> - -If you do not know the name of the specific numpy module you wish to -load, you can use the ``spack module tcl|lmod loads`` command to get -the name of the module from the Spack spec. - ----------------------- Filesystem requirements ----------------------- diff --git a/lib/spack/docs/build_systems/pythonpackage.rst b/lib/spack/docs/build_systems/pythonpackage.rst index 372d4ad47c..9512b08885 100644 --- a/lib/spack/docs/build_systems/pythonpackage.rst +++ b/lib/spack/docs/build_systems/pythonpackage.rst @@ -718,23 +718,45 @@ command-line tool, or C/C++/Fortran program with optional Python modules? The former should be prepended with ``py-``, while the latter should not. -"""""""""""""""""""""" -extends vs. depends_on -"""""""""""""""""""""" +"""""""""""""""""""""""""""""" +``extends`` vs. ``depends_on`` +"""""""""""""""""""""""""""""" -This is very similar to the naming dilemma above, with a slight twist. As mentioned in the :ref:`Packaging Guide <packaging_extensions>`, ``extends`` and ``depends_on`` are very similar, but ``extends`` ensures that the extension and extendee share the same prefix in views. This allows the user to import a Python module without having to add that module to ``PYTHONPATH``. -When deciding between ``extends`` and ``depends_on``, the best rule of -thumb is to check the installation prefix. If Python libraries are -installed to ``<prefix>/lib/pythonX.Y/site-packages``, then you -should use ``extends``. If Python libraries are installed elsewhere -or the only files that get installed reside in ``<prefix>/bin``, then -don't use ``extends``. +Additionally, ``extends("python")`` adds a dependency on the package +``python-venv``. This improves isolation from the system, whether +it's during the build or at runtime: user and system site packages +cannot accidentally be used by any package that ``extends("python")``. + +As a rule of thumb: if a package does not install any Python modules +of its own, and merely puts a Python script in the ``bin`` directory, +then there is no need for ``extends``. If the package installs modules +in the ``site-packages`` directory, it requires ``extends``. + +""""""""""""""""""""""""""""""""""""" +Executing ``python`` during the build +""""""""""""""""""""""""""""""""""""" + +Whenever you need to execute a Python command or pass the path of the +Python interpreter to the build system, it is best to use the global +variable ``python`` directly. For example: + +.. code-block:: python + + @run_before("install") + def recythonize(self): + python("setup.py", "clean") # use the `python` global + +As mentioned in the previous section, ``extends("python")`` adds an +automatic dependency on ``python-venv``, which is a virtual environment +that guarantees build isolation. The ``python`` global always refers to +the correct Python interpreter, whether the package uses ``extends("python")`` +or ``depends_on("python")``. ^^^^^^^^^^^^^^^^^^^^^ Alternatives to Spack diff --git a/lib/spack/spack/bootstrap/_common.py b/lib/spack/spack/bootstrap/_common.py index 2ce53d3165..5c3ca93e94 100644 --- a/lib/spack/spack/bootstrap/_common.py +++ b/lib/spack/spack/bootstrap/_common.py @@ -54,10 +54,14 @@ def _try_import_from_store( installed_specs = spack.store.STORE.db.query(query_spec, installed=True) for candidate_spec in installed_specs: - pkg = candidate_spec["python"].package + # previously bootstrapped specs may not have a python-venv dependency. + if candidate_spec.dependencies("python-venv"): + python, *_ = candidate_spec.dependencies("python-venv") + else: + python, *_ = candidate_spec.dependencies("python") module_paths = [ - os.path.join(candidate_spec.prefix, pkg.purelib), - os.path.join(candidate_spec.prefix, pkg.platlib), + os.path.join(candidate_spec.prefix, python.package.purelib), + os.path.join(candidate_spec.prefix, python.package.platlib), ] path_before = list(sys.path) diff --git a/lib/spack/spack/bootstrap/environment.py b/lib/spack/spack/bootstrap/environment.py index f1af8990e8..13942ba86f 100644 --- a/lib/spack/spack/bootstrap/environment.py +++ b/lib/spack/spack/bootstrap/environment.py @@ -3,13 +3,11 @@ # # SPDX-License-Identifier: (Apache-2.0 OR MIT) """Bootstrap non-core Spack dependencies from an environment.""" -import glob import hashlib import os import pathlib import sys -import warnings -from typing import List +from typing import Iterable, List import archspec.cpu @@ -28,6 +26,16 @@ from .core import _add_externals_if_missing class BootstrapEnvironment(spack.environment.Environment): """Environment to install dependencies of Spack for a given interpreter and architecture""" + def __init__(self) -> None: + if not self.spack_yaml().exists(): + self._write_spack_yaml_file() + super().__init__(self.environment_root()) + + # Remove python package roots created before python-venv was introduced + for s in self.concrete_roots(): + if "python" in s.package.extendees and not s.dependencies("python-venv"): + self.deconcretize(s) + @classmethod def spack_dev_requirements(cls) -> List[str]: """Spack development requirements""" @@ -59,31 +67,19 @@ class BootstrapEnvironment(spack.environment.Environment): return cls.environment_root().joinpath("view") @classmethod - def pythonpaths(cls) -> List[str]: - """Paths to be added to sys.path or PYTHONPATH""" - python_dir_part = f"python{'.'.join(str(x) for x in sys.version_info[:2])}" - glob_expr = str(cls.view_root().joinpath("**", python_dir_part, "**")) - result = glob.glob(glob_expr) - if not result: - msg = f"Cannot find any Python path in {cls.view_root()}" - warnings.warn(msg) - return result - - @classmethod - def bin_dirs(cls) -> List[pathlib.Path]: + def bin_dir(cls) -> pathlib.Path: """Paths to be added to PATH""" - return [cls.view_root().joinpath("bin")] + return cls.view_root().joinpath("bin") + + def python_dirs(self) -> Iterable[pathlib.Path]: + python = next(s for s in self.all_specs_generator() if s.name == "python-venv").package + return {self.view_root().joinpath(p) for p in (python.platlib, python.purelib)} @classmethod def spack_yaml(cls) -> pathlib.Path: """Environment spack.yaml file""" return cls.environment_root().joinpath("spack.yaml") - def __init__(self) -> None: - if not self.spack_yaml().exists(): - self._write_spack_yaml_file() - super().__init__(self.environment_root()) - def update_installations(self) -> None: """Update the installations of this environment.""" log_enabled = tty.is_debug() or tty.is_verbose() @@ -100,21 +96,13 @@ class BootstrapEnvironment(spack.environment.Environment): self.install_all() self.write(regenerate=True) - def update_syspath_and_environ(self) -> None: - """Update ``sys.path`` and the PATH, PYTHONPATH environment variables to point to - the environment view. - """ - # Do minimal modifications to sys.path and environment variables. In particular, pay - # attention to have the smallest PYTHONPATH / sys.path possible, since that may impact - # the performance of the current interpreter - sys.path.extend(self.pythonpaths()) - os.environ["PATH"] = os.pathsep.join( - [str(x) for x in self.bin_dirs()] + os.environ.get("PATH", "").split(os.pathsep) - ) - os.environ["PYTHONPATH"] = os.pathsep.join( - os.environ.get("PYTHONPATH", "").split(os.pathsep) - + [str(x) for x in self.pythonpaths()] - ) + def load(self) -> None: + """Update PATH and sys.path.""" + # Make executables available (shouldn't need PYTHONPATH) + os.environ["PATH"] = f"{self.bin_dir()}{os.pathsep}{os.environ.get('PATH', '')}" + + # Spack itself imports pytest + sys.path.extend(str(p) for p in self.python_dirs()) def _write_spack_yaml_file(self) -> None: tty.msg( @@ -164,4 +152,4 @@ def ensure_environment_dependencies() -> None: _add_externals_if_missing() with BootstrapEnvironment() as env: env.update_installations() - env.update_syspath_and_environ() + env.load() diff --git a/lib/spack/spack/build_systems/cmake.py b/lib/spack/spack/build_systems/cmake.py index b6e66e136c..a64904715e 100644 --- a/lib/spack/spack/build_systems/cmake.py +++ b/lib/spack/spack/build_systems/cmake.py @@ -39,16 +39,11 @@ def _maybe_set_python_hints(pkg: spack.package_base.PackageBase, args: List[str] """Set the PYTHON_EXECUTABLE, Python_EXECUTABLE, and Python3_EXECUTABLE CMake variables if the package has Python as build or link dep and ``find_python_hints`` is set to True. See ``find_python_hints`` for context.""" - if not getattr(pkg, "find_python_hints", False): + if not getattr(pkg, "find_python_hints", False) or not pkg.spec.dependencies( + "python", dt.BUILD | dt.LINK + ): return - pythons = pkg.spec.dependencies("python", dt.BUILD | dt.LINK) - if len(pythons) != 1: - return - try: - python_executable = pythons[0].package.command.path - except RuntimeError: - return - + python_executable = pkg.spec["python"].command.path args.extend( [ CMakeBuilder.define("PYTHON_EXECUTABLE", python_executable), diff --git a/lib/spack/spack/build_systems/python.py b/lib/spack/spack/build_systems/python.py index 1f650be98a..c94e2db700 100644 --- a/lib/spack/spack/build_systems/python.py +++ b/lib/spack/spack/build_systems/python.py @@ -120,6 +120,12 @@ class PythonExtension(spack.package_base.PackageBase): """ return [] + @property + def python_spec(self): + """Get python-venv if it exists or python otherwise.""" + python, *_ = self.spec.dependencies("python-venv") or self.spec.dependencies("python") + return python + def view_file_conflicts(self, view, merge_map): """Report all file conflicts, excepting special cases for python. Specifically, this does not report errors for duplicate @@ -138,16 +144,17 @@ class PythonExtension(spack.package_base.PackageBase): return conflicts def add_files_to_view(self, view, merge_map, skip_if_exists=True): - # Patch up shebangs to the python linked in the view only if python is built by Spack. - if not self.extendee_spec or self.extendee_spec.external: + # Patch up shebangs if the package extends Python and we put a Python interpreter in the + # view. + python = self.python_spec + if not self.extendee_spec or python.external: return super().add_files_to_view(view, merge_map, skip_if_exists) # We only patch shebangs in the bin directory. copied_files: Dict[Tuple[int, int], str] = {} # File identifier -> source delayed_links: List[Tuple[str, str]] = [] # List of symlinks from merge map - bin_dir = self.spec.prefix.bin - python_prefix = self.extendee_spec.prefix + for src, dst in merge_map.items(): if skip_if_exists and os.path.lexists(dst): continue @@ -168,7 +175,7 @@ class PythonExtension(spack.package_base.PackageBase): copied_files[(s.st_dev, s.st_ino)] = dst shutil.copy2(src, dst) fs.filter_file( - python_prefix, os.path.abspath(view.get_projection_for_spec(self.spec)), dst + python.prefix, os.path.abspath(view.get_projection_for_spec(self.spec)), dst ) else: view.link(src, dst) @@ -199,14 +206,13 @@ class PythonExtension(spack.package_base.PackageBase): ignore_namespace = True bin_dir = self.spec.prefix.bin - global_view = self.extendee_spec.prefix == view.get_projection_for_spec(self.spec) to_remove = [] for src, dst in merge_map.items(): if ignore_namespace and namespace_init(dst): continue - if global_view or not fs.path_contains_subdirectory(src, bin_dir): + if not fs.path_contains_subdirectory(src, bin_dir): to_remove.append(dst) else: os.remove(dst) @@ -371,8 +377,9 @@ class PythonPackage(PythonExtension): # Headers should only be in include or platlib, but no harm in checking purelib too include = self.prefix.join(self.spec["python"].package.include).join(name) - platlib = self.prefix.join(self.spec["python"].package.platlib).join(name) - purelib = self.prefix.join(self.spec["python"].package.purelib).join(name) + python = self.python_spec + platlib = self.prefix.join(python.package.platlib).join(name) + purelib = self.prefix.join(python.package.purelib).join(name) headers_list = map(fs.find_all_headers, [include, platlib, purelib]) headers = functools.reduce(operator.add, headers_list) @@ -391,8 +398,9 @@ class PythonPackage(PythonExtension): name = self.spec.name[3:] # Libraries should only be in platlib, but no harm in checking purelib too - platlib = self.prefix.join(self.spec["python"].package.platlib).join(name) - purelib = self.prefix.join(self.spec["python"].package.purelib).join(name) + python = self.python_spec + platlib = self.prefix.join(python.package.platlib).join(name) + purelib = self.prefix.join(python.package.purelib).join(name) find_all_libraries = functools.partial(fs.find_all_libraries, recursive=True) libs_list = map(find_all_libraries, [platlib, purelib]) @@ -504,6 +512,8 @@ class PythonPipBuilder(BaseBuilder): def install(self, pkg: PythonPackage, spec: Spec, prefix: Prefix) -> None: """Install everything from build directory.""" + pip = spec["python"].command + pip.add_default_arg("-m", "pip") args = PythonPipBuilder.std_args(pkg) + [f"--prefix={prefix}"] @@ -519,14 +529,6 @@ class PythonPipBuilder(BaseBuilder): else: args.append(".") - pip = spec["python"].command - # Hide user packages, since we don't have build isolation. This is - # necessary because pip / setuptools may run hooks from arbitrary - # packages during the build. There is no equivalent variable to hide - # system packages, so this is not reliable for external Python. - pip.add_default_env("PYTHONNOUSERSITE", "1") - pip.add_default_arg("-m") - pip.add_default_arg("pip") with fs.working_dir(self.build_directory): pip(*args) diff --git a/lib/spack/spack/directives.py b/lib/spack/spack/directives.py index 4991040142..b69f83a75d 100644 --- a/lib/spack/spack/directives.py +++ b/lib/spack/spack/directives.py @@ -662,6 +662,7 @@ def _execute_redistribute( @directive(("extendees", "dependencies")) def extends(spec, when=None, type=("build", "run"), patches=None): """Same as depends_on, but also adds this package to the extendee list. + In case of Python, also adds a dependency on python-venv. keyword arguments can be passed to extends() so that extension packages can pass parameters to the extendee's extension @@ -677,6 +678,11 @@ def extends(spec, when=None, type=("build", "run"), patches=None): _depends_on(pkg, spec, when=when, type=type, patches=patches) spec_obj = spack.spec.Spec(spec) + # When extending python, also add a dependency on python-venv. This is done so that + # Spack environment views are Python virtual environments. + if spec_obj.name == "python" and not pkg.name == "python-venv": + _depends_on(pkg, "python-venv", when=when, type=("build", "run")) + # TODO: the values of the extendees dictionary are not used. Remove in next refactor. pkg.extendees[spec_obj.name] = (spec_obj, None) diff --git a/lib/spack/spack/hooks/windows_runtime_linkage.py b/lib/spack/spack/hooks/windows_runtime_linkage.py new file mode 100644 index 0000000000..5bb3744910 --- /dev/null +++ b/lib/spack/spack/hooks/windows_runtime_linkage.py @@ -0,0 +1,8 @@ +# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other +# Spack Project Developers. See the top-level COPYRIGHT file for details. +# +# SPDX-License-Identifier: (Apache-2.0 OR MIT) + + +def post_install(spec, explicit=None): + spec.package.windows_establish_runtime_linkage() diff --git a/lib/spack/spack/installer.py b/lib/spack/spack/installer.py index 1f33a7c6b0..289a48568d 100644 --- a/lib/spack/spack/installer.py +++ b/lib/spack/spack/installer.py @@ -1698,10 +1698,6 @@ class PackageInstaller: spack.package_base.PackageBase._verbose = spack.build_environment.start_build_process( pkg, build_process, install_args ) - # Currently this is how RPATH-like behavior is achieved on Windows, after install - # establish runtime linkage via Windows Runtime link object - # Note: this is a no-op on non Windows platforms - pkg.windows_establish_runtime_linkage() # Note: PARENT of the build process adds the new package to # the database, so that we don't need to re-read from file. spack.store.STORE.db.add(pkg.spec, spack.store.STORE.layout, explicit=explicit) diff --git a/lib/spack/spack/spec.py b/lib/spack/spack/spec.py index 1f30f7e923..27c762bd40 100644 --- a/lib/spack/spack/spec.py +++ b/lib/spack/spack/spec.py @@ -1030,16 +1030,13 @@ class _EdgeMap(collections.abc.Mapping): self.edges.clear() -def _command_default_handler(descriptor, spec, cls): +def _command_default_handler(spec: "Spec"): """Default handler when looking for the 'command' attribute. Tries to search for ``spec.name`` in the ``spec.home.bin`` directory. Parameters: - descriptor (ForwardQueryToPackage): descriptor that triggered the call - spec (Spec): spec that is being queried - cls (type(spec)): type of spec, to match the signature of the - descriptor ``__get__`` method + spec: spec that is being queried Returns: Executable: An executable of the command @@ -1052,22 +1049,17 @@ def _command_default_handler(descriptor, spec, cls): if fs.is_exe(path): return spack.util.executable.Executable(path) - else: - msg = "Unable to locate {0} command in {1}" - raise RuntimeError(msg.format(spec.name, home.bin)) + raise RuntimeError(f"Unable to locate {spec.name} command in {home.bin}") -def _headers_default_handler(descriptor, spec, cls): +def _headers_default_handler(spec: "Spec"): """Default handler when looking for the 'headers' attribute. Tries to search for ``*.h`` files recursively starting from ``spec.package.home.include``. Parameters: - descriptor (ForwardQueryToPackage): descriptor that triggered the call - spec (Spec): spec that is being queried - cls (type(spec)): type of spec, to match the signature of the - descriptor ``__get__`` method + spec: spec that is being queried Returns: HeaderList: The headers in ``prefix.include`` @@ -1080,12 +1072,10 @@ def _headers_default_handler(descriptor, spec, cls): if headers: return headers - else: - msg = "Unable to locate {0} headers in {1}" - raise spack.error.NoHeadersError(msg.format(spec.name, home)) + raise spack.error.NoHeadersError(f"Unable to locate {spec.name} headers in {home}") -def _libs_default_handler(descriptor, spec, cls): +def _libs_default_handler(spec: "Spec"): """Default handler when looking for the 'libs' attribute. Tries to search for ``lib{spec.name}`` recursively starting from @@ -1093,10 +1083,7 @@ def _libs_default_handler(descriptor, spec, cls): ``{spec.name}`` instead. Parameters: - descriptor (ForwardQueryToPackage): descriptor that triggered the call - spec (Spec): spec that is being queried - cls (type(spec)): type of spec, to match the signature of the - descriptor ``__get__`` method + spec: spec that is being queried Returns: LibraryList: The libraries found @@ -1135,27 +1122,33 @@ def _libs_default_handler(descriptor, spec, cls): if libs: return libs - msg = "Unable to recursively locate {0} libraries in {1}" - raise spack.error.NoLibrariesError(msg.format(spec.name, home)) + raise spack.error.NoLibrariesError( + f"Unable to recursively locate {spec.name} libraries in {home}" + ) class ForwardQueryToPackage: """Descriptor used to forward queries from Spec to Package""" - def __init__(self, attribute_name, default_handler=None): + def __init__( + self, + attribute_name: str, + default_handler: Optional[Callable[["Spec"], Any]] = None, + _indirect: bool = False, + ) -> None: """Create a new descriptor. Parameters: - attribute_name (str): name of the attribute to be - searched for in the Package instance - default_handler (callable, optional): default function to be - called if the attribute was not found in the Package - instance + attribute_name: name of the attribute to be searched for in the Package instance + default_handler: default function to be called if the attribute was not found in the + Package instance + _indirect: temporarily added to redirect a query to another package. """ self.attribute_name = attribute_name self.default = default_handler + self.indirect = _indirect - def __get__(self, instance, cls): + def __get__(self, instance: "SpecBuildInterface", cls): """Retrieves the property from Package using a well defined chain of responsibility. @@ -1177,13 +1170,18 @@ class ForwardQueryToPackage: indicating a query failure, e.g. that library files were not found in a 'libs' query. """ - pkg = instance.package + # TODO: this indirection exist solely for `spec["python"].command` to actually return + # spec["python-venv"].command. It should be removed when `python` is a virtual. + if self.indirect and instance.indirect_spec: + pkg = instance.indirect_spec.package + else: + pkg = instance.wrapped_obj.package try: query = instance.last_query except AttributeError: # There has been no query yet: this means # a spec is trying to access its own attributes - _ = instance[instance.name] # NOQA: ignore=F841 + _ = instance.wrapped_obj[instance.wrapped_obj.name] # NOQA: ignore=F841 query = instance.last_query callbacks_chain = [] @@ -1195,7 +1193,8 @@ class ForwardQueryToPackage: callbacks_chain.append(lambda: getattr(pkg, self.attribute_name)) # Final resort : default callback if self.default is not None: - callbacks_chain.append(lambda: self.default(self, instance, cls)) + _default = self.default # make mypy happy + callbacks_chain.append(lambda: _default(instance.wrapped_obj)) # Trigger the callbacks in order, the first one producing a # value wins @@ -1254,25 +1253,33 @@ QueryState = collections.namedtuple("QueryState", ["name", "extra_parameters", " class SpecBuildInterface(lang.ObjectWrapper): # home is available in the base Package so no default is needed home = ForwardQueryToPackage("home", default_handler=None) - - command = ForwardQueryToPackage("command", default_handler=_command_default_handler) - headers = ForwardQueryToPackage("headers", default_handler=_headers_default_handler) - libs = ForwardQueryToPackage("libs", default_handler=_libs_default_handler) + command = ForwardQueryToPackage( + "command", default_handler=_command_default_handler, _indirect=True + ) - def __init__(self, spec, name, query_parameters): + def __init__(self, spec: "Spec", name: str, query_parameters: List[str], _parent: "Spec"): super().__init__(spec) # Adding new attributes goes after super() call since the ObjectWrapper # resets __dict__ to behave like the passed object original_spec = getattr(spec, "wrapped_obj", spec) self.wrapped_obj = original_spec - self.token = original_spec, name, query_parameters + self.token = original_spec, name, query_parameters, _parent is_virtual = spack.repo.PATH.is_virtual(name) self.last_query = QueryState( name=name, extra_parameters=query_parameters, isvirtual=is_virtual ) + # TODO: this ad-hoc logic makes `spec["python"].command` return + # `spec["python-venv"].command` and should be removed when `python` is a virtual. + self.indirect_spec = None + if spec.name == "python": + python_venvs = _parent.dependencies("python-venv") + if not python_venvs: + return + self.indirect_spec = python_venvs[0] + def __reduce__(self): return SpecBuildInterface, self.token @@ -4137,7 +4144,7 @@ class Spec: raise spack.error.SpecError("Spec version is not concrete: " + str(self)) return self.versions[0] - def __getitem__(self, name): + def __getitem__(self, name: str): """Get a dependency from the spec by its name. This call implicitly sets a query state in the package being retrieved. The behavior of packages may be influenced by additional query parameters that are @@ -4146,7 +4153,7 @@ class Spec: Note that if a virtual package is queried a copy of the Spec is returned while for non-virtual a reference is returned. """ - query_parameters = name.split(":") + query_parameters: List[str] = name.split(":") if len(query_parameters) > 2: raise KeyError("key has more than one ':' symbol. At most one is admitted.") @@ -4169,7 +4176,7 @@ class Spec: ) try: - value = next( + child: Spec = next( itertools.chain( # Regular specs (x for x in order() if x.name == name), @@ -4186,9 +4193,9 @@ class Spec: raise KeyError(f"No spec with name {name} in {self}") if self._concrete: - return SpecBuildInterface(value, name, query_parameters) + return SpecBuildInterface(child, name, query_parameters, _parent=self) - return value + return child def __contains__(self, spec): """True if this spec or some dependency satisfies the spec. diff --git a/lib/spack/spack/test/cmd/extensions.py b/lib/spack/spack/test/cmd/extensions.py index 1f6ed95b56..5869e46642 100644 --- a/lib/spack/spack/test/cmd/extensions.py +++ b/lib/spack/spack/test/cmd/extensions.py @@ -33,21 +33,23 @@ def test_extensions(mock_packages, python_database, config, capsys): packages = extensions("-s", "packages", "python") installed = extensions("-s", "installed", "python") assert "==> python@2.7.11" in output - assert "==> 2 extensions" in output + assert "==> 3 extensions" in output assert "py-extension1" in output assert "py-extension2" in output + assert "python-venv" in output - assert "==> 2 extensions" in packages + assert "==> 3 extensions" in packages assert "py-extension1" in packages assert "py-extension2" in packages + assert "python-venv" in packages assert "installed" not in packages - assert ("%s installed" % (ni if ni else "None")) in output - assert ("%s installed" % (ni if ni else "None")) in installed + assert f"{ni if ni else 'None'} installed" in output + assert f"{ni if ni else 'None'} installed" in installed - check_output(2) + check_output(3) ext2.package.do_uninstall(force=True) - check_output(1) + check_output(2) def test_extensions_no_arguments(mock_packages): diff --git a/lib/spack/spack/util/executable.py b/lib/spack/spack/util/executable.py index f160051674..afb8bcaa39 100644 --- a/lib/spack/spack/util/executable.py +++ b/lib/spack/spack/util/executable.py @@ -39,6 +39,20 @@ class Executable: """Add default argument(s) to the command.""" self.exe.extend(args) + def with_default_args(self, *args): + """Same as add_default_arg, but returns a copy of the executable.""" + new = self.copy() + new.add_default_arg(*args) + return new + + def copy(self): + """Return a copy of this Executable.""" + new = Executable(self.exe[0]) + new.exe[:] = self.exe + new.default_env.update(self.default_env) + new.default_envmod.extend(self.default_envmod) + return new + def add_default_env(self, key, value): """Set an environment variable when the command is run. diff --git a/var/spack/repos/builtin.mock/packages/python-venv/package.py b/var/spack/repos/builtin.mock/packages/python-venv/package.py new file mode 100644 index 0000000000..741fc3c627 --- /dev/null +++ b/var/spack/repos/builtin.mock/packages/python-venv/package.py @@ -0,0 +1,21 @@ +# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other +# Spack Project Developers. See the top-level COPYRIGHT file for details. +# +# SPDX-License-Identifier: (Apache-2.0 OR MIT) + + +from spack.package import * + + +class PythonVenv(Package): + """A Spack managed Python virtual environment""" + + homepage = "https://docs.python.org/3/library/venv.html" + has_code = False + + version("1.0") + + extends("python") + + def install(self, spec, prefix): + pass diff --git a/var/spack/repos/builtin/packages/ascent/package.py b/var/spack/repos/builtin/packages/ascent/package.py index 5fb06916f3..17c5604033 100644 --- a/var/spack/repos/builtin/packages/ascent/package.py +++ b/var/spack/repos/builtin/packages/ascent/package.py @@ -495,7 +495,7 @@ class Ascent(CMakePackage, CudaPackage): cfg.write("# Enable python module builds\n") cfg.write(cmake_cache_entry("ENABLE_PYTHON", "ON")) cfg.write("# python from spack \n") - cfg.write(cmake_cache_entry("PYTHON_EXECUTABLE", spec["python"].command.path)) + cfg.write(cmake_cache_entry("PYTHON_EXECUTABLE", python.path)) try: cfg.write("# python module install dir\n") cfg.write(cmake_cache_entry("PYTHON_MODULE_INSTALL_PREFIX", python_platlib)) diff --git a/var/spack/repos/builtin/packages/bohrium/package.py b/var/spack/repos/builtin/packages/bohrium/package.py index 64ad8c100e..fd9e665663 100644 --- a/var/spack/repos/builtin/packages/bohrium/package.py +++ b/var/spack/repos/builtin/packages/bohrium/package.py @@ -256,10 +256,6 @@ class Bohrium(CMakePackage, CudaPackage): cxx("-o", "test_cxxadd", file_cxxadd, *cxx_flags) test_cxxadd = Executable("./test_cxxadd") - # Build python test commandline - file_pyadd = join_path(os.path.dirname(self.module.__file__), "pyadd.py") - test_pyadd = Executable(spec["python"].command.path + " " + file_pyadd) - # Run tests for each available stack for bh_stack in stacks: tty.info("Testing with bohrium stack '" + bh_stack + "'") @@ -270,5 +266,6 @@ class Bohrium(CMakePackage, CudaPackage): # Python test (if +python) if "+python" in spec: - py_output = test_pyadd(output=str, env=test_env) + file_pyadd = join_path(os.path.dirname(self.module.__file__), "pyadd.py") + py_output = python(file_pyadd, output=str, env=test_env) compare_output(py_output, "Success!\n") diff --git a/var/spack/repos/builtin/packages/cantera/package.py b/var/spack/repos/builtin/packages/cantera/package.py index efb9d7fcbe..ef057f955b 100644 --- a/var/spack/repos/builtin/packages/cantera/package.py +++ b/var/spack/repos/builtin/packages/cantera/package.py @@ -134,13 +134,9 @@ class Cantera(SConsPackage): # Python module if "+python" in spec: - args.extend( - ["python_package=full", "python_cmd={0}".format(spec["python"].command.path)] - ) + args.extend(["python_package=full", "python_cmd={0}".format(python.path)]) if spec["python"].satisfies("@3:"): - args.extend( - ["python3_package=y", "python3_cmd={0}".format(spec["python"].command.path)] - ) + args.extend(["python3_package=y", "python3_cmd={0}".format(python.path)]) else: args.append("python3_package=n") else: diff --git a/var/spack/repos/builtin/packages/clingo-bootstrap/package.py b/var/spack/repos/builtin/packages/clingo-bootstrap/package.py index 022cb7e6e3..abb5c53b3c 100644 --- a/var/spack/repos/builtin/packages/clingo-bootstrap/package.py +++ b/var/spack/repos/builtin/packages/clingo-bootstrap/package.py @@ -120,9 +120,7 @@ class ClingoBootstrap(Clingo): ) python_runtime_env.unset("SPACK_ENV") python_runtime_env.unset("SPACK_PYTHON") - self.spec["python"].command( - spack.paths.spack_script, "solve", "--fresh", "hdf5", extra_env=python_runtime_env - ) + python(spack.paths.spack_script, "solve", "--fresh", "hdf5", extra_env=python_runtime_env) # Clean the build dir. rmtree(self.build_directory, ignore_errors=True) diff --git a/var/spack/repos/builtin/packages/clingo/package.py b/var/spack/repos/builtin/packages/clingo/package.py index fe1a6f859f..7af9b213fb 100644 --- a/var/spack/repos/builtin/packages/clingo/package.py +++ b/var/spack/repos/builtin/packages/clingo/package.py @@ -3,8 +3,6 @@ # # SPDX-License-Identifier: (Apache-2.0 OR MIT) -import os - from spack.compiler import UnsupportedCompilerFlag from spack.package import * @@ -123,7 +121,4 @@ class Clingo(CMakePackage): return args def win_add_library_dependent(self): - if "+python" in self.spec: - return [os.path.join(self.prefix, self.spec["python"].package.platlib)] - else: - return [] + return [python_platlib] if "+python" in self.spec else [] diff --git a/var/spack/repos/builtin/packages/conduit/package.py b/var/spack/repos/builtin/packages/conduit/package.py index c1066ce752..001b04953d 100644 --- a/var/spack/repos/builtin/packages/conduit/package.py +++ b/var/spack/repos/builtin/packages/conduit/package.py @@ -443,7 +443,7 @@ class Conduit(CMakePackage): cfg.write("# Enable python module builds\n") cfg.write(cmake_cache_entry("ENABLE_PYTHON", "ON")) cfg.write("# python from spack \n") - cfg.write(cmake_cache_entry("PYTHON_EXECUTABLE", spec["python"].command.path)) + cfg.write(cmake_cache_entry("PYTHON_EXECUTABLE", python.path)) try: cfg.write("# python module install dir\n") cfg.write(cmake_cache_entry("PYTHON_MODULE_INSTALL_PREFIX", python_platlib)) diff --git a/var/spack/repos/builtin/packages/dsqss/package.py b/var/spack/repos/builtin/packages/dsqss/package.py index 72e5f5961f..44f76ab619 100644 --- a/var/spack/repos/builtin/packages/dsqss/package.py +++ b/var/spack/repos/builtin/packages/dsqss/package.py @@ -58,7 +58,6 @@ class Dsqss(CMakePackage): copy(join_path(test01, "std.toml"), ".") # prepare - python = self.spec["python"].command opts = [self.spec.prefix.bin.dla_pre, "std.toml"] with test_part(self, "test_dla_pre", purpose="prepare dla"): python(*opts) diff --git a/var/spack/repos/builtin/packages/fenics/package.py b/var/spack/repos/builtin/packages/fenics/package.py index ae8336aa39..cf56c14fd0 100644 --- a/var/spack/repos/builtin/packages/fenics/package.py +++ b/var/spack/repos/builtin/packages/fenics/package.py @@ -156,7 +156,7 @@ class Fenics(CMakePackage): depends_on("py-sphinx@1.0.1:", when="+doc", type="build") def cmake_args(self): - args = [ + return [ self.define_from_variant("BUILD_SHARED_LIBS", "shared"), self.define("DOLFIN_SKIP_BUILD_TESTS", True), self.define_from_variant("DOLFIN_ENABLE_OPENMP", "openmp"), @@ -180,11 +180,6 @@ class Fenics(CMakePackage): self.define_from_variant("DOLFIN_ENABLE_ZLIB", "zlib"), ] - if "+python" in self.spec: - args.append(self.define("PYTHON_EXECUTABLE", self.spec["python"].command.path)) - - return args - # set environment for bulding python interface def setup_build_environment(self, env): env.set("DOLFIN_DIR", self.prefix) diff --git a/var/spack/repos/builtin/packages/gurobi/package.py b/var/spack/repos/builtin/packages/gurobi/package.py index c0fa33639d..d9dd35c2ed 100644 --- a/var/spack/repos/builtin/packages/gurobi/package.py +++ b/var/spack/repos/builtin/packages/gurobi/package.py @@ -57,5 +57,4 @@ class Gurobi(Package): @run_after("install") def gurobipy(self): with working_dir("linux64"): - python = which("python") python("setup.py", "install", "--prefix={0}".format(self.prefix)) diff --git a/var/spack/repos/builtin/packages/lbann/package.py b/var/spack/repos/builtin/packages/lbann/package.py index 482d591a1c..e875832b92 100644 --- a/var/spack/repos/builtin/packages/lbann/package.py +++ b/var/spack/repos/builtin/packages/lbann/package.py @@ -402,12 +402,8 @@ class Lbann(CachedCMakePackage, CudaPackage, ROCmPackage): ) entries.append(cmake_cache_option("protobuf_MODULE_COMPATIBLE", True)) - if spec.satisfies("^python") and "+pfe" in spec: - entries.append( - cmake_cache_path( - "LBANN_PFE_PYTHON_EXECUTABLE", "{0}/python3".format(spec["python"].prefix.bin) - ) - ) + if spec.satisfies("+pfe ^python"): + entries.append(cmake_cache_path("LBANN_PFE_PYTHON_EXECUTABLE", python.path)) entries.append( cmake_cache_string("LBANN_PFE_PYTHONPATH", env["PYTHONPATH"]) ) # do NOT need to sub ; for : because diff --git a/var/spack/repos/builtin/packages/libcap-ng/package.py b/var/spack/repos/builtin/packages/libcap-ng/package.py index 373cf7c399..8256fc16a5 100644 --- a/var/spack/repos/builtin/packages/libcap-ng/package.py +++ b/var/spack/repos/builtin/packages/libcap-ng/package.py @@ -33,7 +33,7 @@ class LibcapNg(AutotoolsPackage): def setup_build_environment(self, env): if self.spec.satisfies("+python"): - env.set("PYTHON", self.spec["python"].command.path) + env.set("PYTHON", python.path) def configure_args(self): args = [] diff --git a/var/spack/repos/builtin/packages/mapserver/package.py b/var/spack/repos/builtin/packages/mapserver/package.py index e26951531e..c55cba0fea 100644 --- a/var/spack/repos/builtin/packages/mapserver/package.py +++ b/var/spack/repos/builtin/packages/mapserver/package.py @@ -3,8 +3,6 @@ # # SPDX-License-Identifier: (Apache-2.0 OR MIT) -import os - from spack.package import * @@ -60,9 +58,7 @@ class Mapserver(CMakePackage): # prefix. This hack patches the CMakeLists.txt for the Python # bindings and hard-wires in the right destination. A bit ugly, # sorry, but I don't speak cmake. - pyversiondir = "python{0}".format(self.spec["python"].version.up_to(2)) - sitepackages = os.path.join(self.spec.prefix.lib, pyversiondir, "site-packages") - filter_file(r"\${PYTHON_SITE_PACKAGES}", sitepackages, "mapscript/python/CMakeLists.txt") + filter_file(r"\${PYTHON_SITE_PACKAGES}", python_platlib, "mapscript/python/CMakeLists.txt") def cmake_args(self): args = [] diff --git a/var/spack/repos/builtin/packages/omnitrace/package.py b/var/spack/repos/builtin/packages/omnitrace/package.py index bc3ed31758..384f8ca2db 100644 --- a/var/spack/repos/builtin/packages/omnitrace/package.py +++ b/var/spack/repos/builtin/packages/omnitrace/package.py @@ -141,7 +141,3 @@ class Omnitrace(CMakePackage): files = glob.glob(pattern) if files: env.set("TAU_MAKEFILE", files[0]) - - def setup_run_environment(self, env): - if "+python" in self.spec: - env.prepend_path("PYTHONPATH", join_path(self.prefix.lib, "python", "site-packages")) diff --git a/var/spack/repos/builtin/packages/open3d/package.py b/var/spack/repos/builtin/packages/open3d/package.py index 2a63de25ca..a8f1beee6b 100644 --- a/var/spack/repos/builtin/packages/open3d/package.py +++ b/var/spack/repos/builtin/packages/open3d/package.py @@ -119,7 +119,7 @@ class Open3d(CMakePackage, CudaPackage): def test(self): if "+python" in self.spec: self.run_test( - self.spec["python"].command.path, + python.path, ["-c", "import open3d"], purpose="checking import of open3d", work_dir="spack-test", diff --git a/var/spack/repos/builtin/packages/opencv/package.py b/var/spack/repos/builtin/packages/opencv/package.py index 8e3f3236bb..64375458a3 100644 --- a/var/spack/repos/builtin/packages/opencv/package.py +++ b/var/spack/repos/builtin/packages/opencv/package.py @@ -1027,14 +1027,13 @@ class Opencv(CMakePackage, CudaPackage): ) # Python - python_exe = spec["python"].command.path python_lib = spec["python"].libs[0] python_include_dir = spec["python"].headers.directories[0] if "+python3" in spec: args.extend( [ - self.define("PYTHON3_EXECUTABLE", python_exe), + self.define("PYTHON3_EXECUTABLE", python.path), self.define("PYTHON3_LIBRARY", python_lib), self.define("PYTHON3_INCLUDE_DIR", python_include_dir), self.define("PYTHON2_EXECUTABLE", ""), diff --git a/var/spack/repos/builtin/packages/openwsman/package.py b/var/spack/repos/builtin/packages/openwsman/package.py index 4be921bad9..9d9870706b 100644 --- a/var/spack/repos/builtin/packages/openwsman/package.py +++ b/var/spack/repos/builtin/packages/openwsman/package.py @@ -31,13 +31,9 @@ class Openwsman(CMakePackage): def patch(self): """Change python install directory.""" if self.spec.satisfies("+python"): - python_spec = self.spec["python"] - python_libdir = join_path( - self.spec.prefix.lib, "python" + str(python_spec.version.up_to(2)), "site-packages" - ) filter_file( "DESTINATION .*", - "DESTINATION {0} )".format(python_libdir), + "DESTINATION {0} )".format(python_platlib), join_path("bindings", "python", "CMakeLists.txt"), ) diff --git a/var/spack/repos/builtin/packages/precice/package.py b/var/spack/repos/builtin/packages/precice/package.py index 48484114fd..6426fa51f3 100644 --- a/var/spack/repos/builtin/packages/precice/package.py +++ b/var/spack/repos/builtin/packages/precice/package.py @@ -191,7 +191,7 @@ class Precice(CMakePackage): python_library = spec["python"].libs[0] python_include = spec["python"].headers.directories[0] numpy_include = join_path( - spec["py-numpy"].prefix, spec["python"].package.platlib, "numpy", "core", "include" + spec["py-numpy"].package.module.python_platlib, "numpy", "core", "include" ) cmake_args.extend( [ diff --git a/var/spack/repos/builtin/packages/py-alphafold/package.py b/var/spack/repos/builtin/packages/py-alphafold/package.py index 674f3ebf10..0269a2abdf 100644 --- a/var/spack/repos/builtin/packages/py-alphafold/package.py +++ b/var/spack/repos/builtin/packages/py-alphafold/package.py @@ -77,7 +77,7 @@ class PyAlphafold(PythonPackage, CudaPackage): @run_after("install") def install_scripts(self): mkdirp(self.prefix.bin) - shebang = "#!{0}\n".format(self.spec["python"].command) + shebang = f"#!{python.path}\n" for fname in glob.glob("run_alphafold*.py"): destfile = join_path(self.prefix.bin, fname) with open(fname, "r") as src: diff --git a/var/spack/repos/builtin/packages/py-chainer/package.py b/var/spack/repos/builtin/packages/py-chainer/package.py index 1f78c76a30..a1ab6819a1 100644 --- a/var/spack/repos/builtin/packages/py-chainer/package.py +++ b/var/spack/repos/builtin/packages/py-chainer/package.py @@ -59,7 +59,7 @@ class PyChainer(PythonPackage): mnist_file = join_path(self.install_test_root.examples.chainermn.mnist, "train_mnist.py") mpirun = which(self.spec["mpi"].prefix.bin.mpirun) - opts = ["-n", "4", self.spec["python"].command.path, mnist_file, "-o", "."] + opts = ["-n", "4", python.path, mnist_file, "-o", "."] env["OMP_NUM_THREADS"] = "4" mpirun(*opts) diff --git a/var/spack/repos/builtin/packages/py-eccodes/package.py b/var/spack/repos/builtin/packages/py-eccodes/package.py index 107cddc03e..b53c545e4a 100644 --- a/var/spack/repos/builtin/packages/py-eccodes/package.py +++ b/var/spack/repos/builtin/packages/py-eccodes/package.py @@ -43,5 +43,4 @@ class PyEccodes(PythonPackage): def test_selfcheck(self): """checking system setup""" - python = self.spec["python"].command python("-m", "eccodes", "selfcheck") diff --git a/var/spack/repos/builtin/packages/py-gmxapi/package.py b/var/spack/repos/builtin/packages/py-gmxapi/package.py index 77a03d6fd4..5869d1ad4b 100644 --- a/var/spack/repos/builtin/packages/py-gmxapi/package.py +++ b/var/spack/repos/builtin/packages/py-gmxapi/package.py @@ -49,5 +49,4 @@ class PyGmxapi(PythonPackage): def install_test(self): with working_dir("spack-test", create=True): # test include helper points to right location - python = self.spec["python"].command python("-m", "pytest", "-x", os.path.join(self.build_directory, "test")) diff --git a/var/spack/repos/builtin/packages/py-gpaw/package.py b/var/spack/repos/builtin/packages/py-gpaw/package.py index 8a94a37b1d..6e4181afbd 100644 --- a/var/spack/repos/builtin/packages/py-gpaw/package.py +++ b/var/spack/repos/builtin/packages/py-gpaw/package.py @@ -59,7 +59,7 @@ class PyGpaw(PythonPackage): python_include = spec["python"].headers.directories[0] numpy_include = join_path( - spec["py-numpy"].prefix, spec["python"].package.platlib, "numpy", "core", "include" + spec["py-numpy"].package.module.python_platlib, "numpy", "core", "include" ) libs = blas.libs + lapack.libs + libxc.libs diff --git a/var/spack/repos/builtin/packages/py-installer/package.py b/var/spack/repos/builtin/packages/py-installer/package.py index 01132b63bb..6ecf2c5b4e 100644 --- a/var/spack/repos/builtin/packages/py-installer/package.py +++ b/var/spack/repos/builtin/packages/py-installer/package.py @@ -36,6 +36,4 @@ class PyInstaller(Package, PythonExtension): python(*args) def setup_dependent_package(self, module, dependent_spec): - installer = dependent_spec["python"].command - installer.add_default_arg("-m", "installer") - setattr(module, "installer", installer) + setattr(module, "installer", python.with_default_args("-m", "installer")) diff --git a/var/spack/repos/builtin/packages/py-ipykernel/package.py b/var/spack/repos/builtin/packages/py-ipykernel/package.py index 2357fbb56d..80490e66f0 100644 --- a/var/spack/repos/builtin/packages/py-ipykernel/package.py +++ b/var/spack/repos/builtin/packages/py-ipykernel/package.py @@ -89,4 +89,4 @@ class PyIpykernel(PythonPackage): @run_after("install") def install_data(self): """install the Jupyter kernel spec""" - self.spec["python"].command("-m", "ipykernel", "install", "--prefix=" + self.prefix) + python("-m", "ipykernel", "install", "--prefix=" + self.prefix) diff --git a/var/spack/repos/builtin/packages/py-libensemble/package.py b/var/spack/repos/builtin/packages/py-libensemble/package.py index cf5cbd6b8d..5b553912ae 100644 --- a/var/spack/repos/builtin/packages/py-libensemble/package.py +++ b/var/spack/repos/builtin/packages/py-libensemble/package.py @@ -100,7 +100,6 @@ class PyLibensemble(PythonPackage): if not os.path.isfile(exe): raise SkipTest(f"{script} is missing") - python = self.spec["python"].command python(exe, "--comms", "local", "--nworkers", "2") def test_uniform_sampling(self): diff --git a/var/spack/repos/builtin/packages/py-nanobind/package.py b/var/spack/repos/builtin/packages/py-nanobind/package.py index 909175140b..6fb598acdc 100644 --- a/var/spack/repos/builtin/packages/py-nanobind/package.py +++ b/var/spack/repos/builtin/packages/py-nanobind/package.py @@ -60,5 +60,5 @@ class PyNanobind(PythonPackage): @property def cmake_prefix_paths(self): - paths = [join_path(self.prefix, self.spec["python"].package.platlib, "nanobind", "cmake")] + paths = [join_path(python_platlib, "nanobind", "cmake")] return paths diff --git a/var/spack/repos/builtin/packages/py-pip/package.py b/var/spack/repos/builtin/packages/py-pip/package.py index 023eab21ec..111e50911b 100644 --- a/var/spack/repos/builtin/packages/py-pip/package.py +++ b/var/spack/repos/builtin/packages/py-pip/package.py @@ -77,7 +77,5 @@ class PyPip(Package, PythonExtension): args.insert(0, os.path.join(whl, "pip")) python(*args) - def setup_dependent_package(self, module, dependent_spec): - pip = dependent_spec["python"].command - pip.add_default_arg("-m", "pip") - setattr(module, "pip", pip) + def setup_dependent_package(self, module, dependent_spec: Spec): + setattr(module, "pip", python.with_default_args("-m", "pip")) diff --git a/var/spack/repos/builtin/packages/py-pybind11/package.py b/var/spack/repos/builtin/packages/py-pybind11/package.py index 1fa9f7beee..f7c298309b 100644 --- a/var/spack/repos/builtin/packages/py-pybind11/package.py +++ b/var/spack/repos/builtin/packages/py-pybind11/package.py @@ -102,7 +102,6 @@ class CMakeBuilder(spack.build_systems.cmake.CMakeBuilder): with working_dir("spack-test", create=True): # test include helper points to right location - python = self.spec["python"].command py_inc = python( "-c", "import pybind11 as py; print(py.get_include())", output=str ).strip() diff --git a/var/spack/repos/builtin/packages/py-pymol/package.py b/var/spack/repos/builtin/packages/py-pymol/package.py index fe9bcce3a8..a4ab0b522d 100644 --- a/var/spack/repos/builtin/packages/py-pymol/package.py +++ b/var/spack/repos/builtin/packages/py-pymol/package.py @@ -58,7 +58,7 @@ class PyPymol(PythonPackage): script = join_path(python_platlib, "pymol", "__init__.py") shebang = "#!/bin/sh\n" - fdata = 'exec {0} {1} "$@"'.format(self.spec["python"].command, script) + fdata = f'exec {python.path} {script} "$@"' with open(fname, "w") as new: new.write(shebang + fdata) set_executable(fname) diff --git a/var/spack/repos/builtin/packages/py-pyqt4/package.py b/var/spack/repos/builtin/packages/py-pyqt4/package.py index 31e481d305..00e27c994e 100644 --- a/var/spack/repos/builtin/packages/py-pyqt4/package.py +++ b/var/spack/repos/builtin/packages/py-pyqt4/package.py @@ -48,7 +48,7 @@ class PyPyqt4(SIPPackage): "--destdir", python_platlib, "--pyuic4-interpreter", - self.spec["python"].command.path, + python.path, "--sipdir", self.prefix.share.sip.PyQt4, "--stubsdir", diff --git a/var/spack/repos/builtin/packages/py-pyspark/package.py b/var/spack/repos/builtin/packages/py-pyspark/package.py index 087378d753..058ac47bf7 100644 --- a/var/spack/repos/builtin/packages/py-pyspark/package.py +++ b/var/spack/repos/builtin/packages/py-pyspark/package.py @@ -25,5 +25,5 @@ class PyPyspark(PythonPackage): depends_on("py-py4j@0.10.9", when="@3.0.1:3.1.3", type=("build", "run")) def setup_run_environment(self, env): - env.set("PYSPARK_PYTHON", self.spec["python"].command.path) - env.set("PYSPARK_DRIVER_PYTHON", self.spec["python"].command.path) + env.set("PYSPARK_PYTHON", python.path) + env.set("PYSPARK_DRIVER_PYTHON", python.path) diff --git a/var/spack/repos/builtin/packages/py-pythonsollya/package.py b/var/spack/repos/builtin/packages/py-pythonsollya/package.py index 84a715c8d1..c8f427f873 100644 --- a/var/spack/repos/builtin/packages/py-pythonsollya/package.py +++ b/var/spack/repos/builtin/packages/py-pythonsollya/package.py @@ -34,9 +34,4 @@ class PyPythonsollya(PythonPackage): @run_before("install") def patch(self): - filter_file( - "PYTHON ?= python2", - "PYTHON ?= " + self.spec["python"].command.path, - "GNUmakefile", - string=True, - ) + filter_file("PYTHON ?= python2", f"PYTHON ?= {python.path}", "GNUmakefile", string=True) diff --git a/var/spack/repos/builtin/packages/py-tensorflow/package.py b/var/spack/repos/builtin/packages/py-tensorflow/package.py index 9a25ea09d2..56d12ff231 100644 --- a/var/spack/repos/builtin/packages/py-tensorflow/package.py +++ b/var/spack/repos/builtin/packages/py-tensorflow/package.py @@ -410,7 +410,7 @@ class PyTensorflow(Package, CudaPackage, ROCmPackage, PythonExtension): spec = self.spec # Please specify the location of python - env.set("PYTHON_BIN_PATH", spec["python"].command.path) + env.set("PYTHON_BIN_PATH", python.path) # Please input the desired Python library path to use env.set("PYTHON_LIB_PATH", python_platlib) diff --git a/var/spack/repos/builtin/packages/py-torch/package.py b/var/spack/repos/builtin/packages/py-torch/package.py index 5de618494d..e7824283fd 100644 --- a/var/spack/repos/builtin/packages/py-torch/package.py +++ b/var/spack/repos/builtin/packages/py-torch/package.py @@ -688,7 +688,5 @@ class PyTorch(PythonPackage, CudaPackage, ROCmPackage): @property def cmake_prefix_paths(self): - cmake_prefix_paths = [ - join_path(self.prefix, self.spec["python"].package.platlib, "torch", "share", "cmake") - ] + cmake_prefix_paths = [join_path(python_platlib, "torch", "share", "cmake")] return cmake_prefix_paths diff --git a/var/spack/repos/builtin/packages/py-xdot/package.py b/var/spack/repos/builtin/packages/py-xdot/package.py index bf898a117e..73d703a287 100644 --- a/var/spack/repos/builtin/packages/py-xdot/package.py +++ b/var/spack/repos/builtin/packages/py-xdot/package.py @@ -50,8 +50,7 @@ class PyXdot(PythonPackage): dst, ) # regenerate the byte-compiled __init__.py - python3 = spec["python"].command - python3("-m", "compileall", dst) + python("-m", "compileall", dst) def setup_run_environment(self, env): spec = self.spec diff --git a/var/spack/repos/builtin/packages/python-venv/package.py b/var/spack/repos/builtin/packages/python-venv/package.py new file mode 100644 index 0000000000..cd0fd84371 --- /dev/null +++ b/var/spack/repos/builtin/packages/python-venv/package.py @@ -0,0 +1,100 @@ +# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other +# Spack Project Developers. See the top-level COPYRIGHT file for details. +# +# SPDX-License-Identifier: (Apache-2.0 OR MIT) + +import os +import shutil + +import llnl.util.filesystem as fs + +from spack.package import * + + +class PythonVenv(Package): + """A Spack managed Python virtual environment""" + + homepage = "https://docs.python.org/3/library/venv.html" + has_code = False + + maintainers("haampie") + + version("1.0") + + extends("python") + + def install(self, spec, prefix): + # Create a virtual environment + python("-m", "venv", "--without-pip", prefix) + + def add_files_to_view(self, view, merge_map: Dict[str, str], skip_if_exists=True): + for src, dst in merge_map.items(): + if skip_if_exists and os.path.lexists(dst): + continue + + name = os.path.basename(dst) + + # Replace the VIRTUAL_ENV variable in the activate scripts after copying + if name.lower().startswith("activate"): + shutil.copy(src, dst) + fs.filter_file( + self.spec.prefix, + os.path.abspath(view.get_projection_for_spec(self.spec)), + dst, + string=True, + ) + else: + view.link(src, dst) + + @property + def bindir(self): + windows = self.spec.satisfies("platform=windows") + return join_path(self.prefix, "Scripts" if windows else "bin") + + @property + def command(self): + """Returns a python Executable instance""" + return which("python3", path=self.bindir) + + def _get_path(self, name) -> str: + return self.command( + "-Ec", f"import sysconfig; print(sysconfig.get_path('{name}'))", output=str + ).strip() + + @property + def platlib(self) -> str: + """Directory for site-specific, platform-specific files.""" + relative_platlib = os.path.relpath(self._get_path("platlib"), self.prefix) + assert not relative_platlib.startswith("..") + return relative_platlib + + @property + def purelib(self) -> str: + """Directory for site-specific, non-platform-specific files.""" + relative_purelib = os.path.relpath(self._get_path("purelib"), self.prefix) + assert not relative_purelib.startswith("..") + return relative_purelib + + @property + def headers(self): + return HeaderList([]) + + @property + def libs(self): + return LibraryList([]) + + def setup_dependent_run_environment(self, env, dependent_spec): + """Set PYTHONPATH to include the site-packages directory for the + extension and any other python extensions it depends on.""" + # Packages may be installed in platform-specific or platform-independent site-packages + # directories + for directory in {self.platlib, self.purelib}: + path = os.path.join(dependent_spec.prefix, directory) + if os.path.isdir(path): + env.prepend_path("PYTHONPATH", path) + + def setup_dependent_package(self, module, dependent_spec): + """Called before python modules' install() methods.""" + module.python = self.command + module.python_platlib = join_path(dependent_spec.prefix, self.platlib) + module.python_purelib = join_path(dependent_spec.prefix, self.purelib) diff --git a/var/spack/repos/builtin/packages/python/package.py b/var/spack/repos/builtin/packages/python/package.py index ad494226ee..b87ee81305 100644 --- a/var/spack/repos/builtin/packages/python/package.py +++ b/var/spack/repos/builtin/packages/python/package.py @@ -8,19 +8,16 @@ import json import os import platform import re -import stat import subprocess import sys from shutil import copy -from typing import Dict, List, Tuple +from typing import Dict, List import llnl.util.tty as tty -from llnl.util.filesystem import is_nonsymlink_exe_with_shebang, path_contains_subdirectory from llnl.util.lang import dedupe from spack.build_environment import dso_suffix, stat_suffix from spack.package import * -from spack.util.environment import is_system_path from spack.util.prefix import Prefix @@ -1115,7 +1112,7 @@ print(json.dumps(config)) path = self.config_vars["platlib"] if path.startswith(prefix): return path.replace(prefix, "") - return os.path.join("lib64", "python{}".format(self.version.up_to(2)), "site-packages") + return os.path.join("lib64", f"python{self.version.up_to(2)}", "site-packages") @property def purelib(self): @@ -1135,7 +1132,7 @@ print(json.dumps(config)) path = self.config_vars["purelib"] if path.startswith(prefix): return path.replace(prefix, "") - return os.path.join("lib", "python{}".format(self.version.up_to(2)), "site-packages") + return os.path.join("lib", f"python{self.version.up_to(2)}", "site-packages") @property def include(self): @@ -1163,39 +1160,6 @@ print(json.dumps(config)) """Set PYTHONPATH to include the site-packages directory for the extension and any other python extensions it depends on. """ - # Ensure the current Python is first in the PATH - path = os.path.dirname(self.command.path) - if not is_system_path(path): - env.prepend_path("PATH", path) - - # Add installation prefix to PYTHONPATH, needed to run import tests - prefixes = set() - if dependent_spec.package.extends(self.spec): - prefixes.add(dependent_spec.prefix) - - # Add direct build/run/test dependencies to PYTHONPATH, - # needed to build the package and to run import tests - for direct_dep in dependent_spec.dependencies(deptype=("build", "run", "test")): - if direct_dep.package.extends(self.spec): - prefixes.add(direct_dep.prefix) - - # Add recursive run dependencies of all direct dependencies, - # needed by direct dependencies at run-time - for indirect_dep in direct_dep.traverse(deptype="run"): - if indirect_dep.package.extends(self.spec): - prefixes.add(indirect_dep.prefix) - - for prefix in prefixes: - # Packages may be installed in platform-specific or platform-independent - # site-packages directories - for directory in {self.platlib, self.purelib}: - env.prepend_path("PYTHONPATH", os.path.join(prefix, directory)) - - if self.spec.satisfies("platform=windows"): - prefix_scripts_dir = prefix.Scripts - if os.path.exists(prefix_scripts_dir): - env.prepend_path("PATH", prefix_scripts_dir) - # We need to make sure that the extensions are compiled and linked with # the Spack wrapper. Paths to the executables that are used for these # operations are normally taken from the sysconfigdata file, which we @@ -1241,9 +1205,7 @@ print(json.dumps(config)) # invoked directly (no change would be required in that case # because Spack arranges for the Spack ld wrapper to be the # first instance of "ld" in PATH). - new_link = config_link.replace( - " {0} ".format(config_compile), " {0} ".format(new_compile) - ) + new_link = config_link.replace(f" {config_compile} ", f" {new_compile} ") # There is logic in the sysconfig module that is sensitive to the # fact that LDSHARED is set in the environment, therefore we export @@ -1256,66 +1218,23 @@ print(json.dumps(config)) """Set PYTHONPATH to include the site-packages directory for the extension and any other python extensions it depends on. """ - if dependent_spec.package.extends(self.spec): - # Packages may be installed in platform-specific or platform-independent - # site-packages directories - for directory in {self.platlib, self.purelib}: - env.prepend_path("PYTHONPATH", os.path.join(dependent_spec.prefix, directory)) + if not dependent_spec.package.extends(self.spec) or dependent_spec.dependencies( + "python-venv" + ): + return + + # Packages may be installed in platform-specific or platform-independent site-packages + # directories + for directory in {self.platlib, self.purelib}: + env.prepend_path("PYTHONPATH", os.path.join(dependent_spec.prefix, directory)) def setup_dependent_package(self, module, dependent_spec): """Called before python modules' install() methods.""" - module.python = self.command - module.python_include = join_path(dependent_spec.prefix, self.include) module.python_platlib = join_path(dependent_spec.prefix, self.platlib) module.python_purelib = join_path(dependent_spec.prefix, self.purelib) - def add_files_to_view(self, view, merge_map, skip_if_exists=True): - # The goal is to copy the `python` executable, so that its search paths are relative to the - # view instead of the install prefix. This is an obsolete way of creating something that - # resembles a virtual environnent. Also we copy scripts with shebang lines. Finally we need - # to re-target symlinks pointing to copied files. - bin_dir = self.spec.prefix.bin if sys.platform != "win32" else self.spec.prefix - copied_files: Dict[Tuple[int, int], str] = {} # File identifier -> source - delayed_links: List[Tuple[str, str]] = [] # List of symlinks from merge map - for src, dst in merge_map.items(): - if skip_if_exists and os.path.lexists(dst): - continue - - # Files not in the bin dir are linked the default way. - if not path_contains_subdirectory(src, bin_dir): - view.link(src, dst, spec=self.spec) - continue - - s = os.lstat(src) - - # Symlink is delayed because we may need to re-target if its target is copied in view - if stat.S_ISLNK(s.st_mode): - delayed_links.append((src, dst)) - continue - - # Anything that's not a symlink gets copied. Scripts with shebangs are immediately - # updated when necessary. - copied_files[(s.st_dev, s.st_ino)] = dst - copy(src, dst) - if is_nonsymlink_exe_with_shebang(src): - filter_file( - self.spec.prefix, os.path.abspath(view.get_projection_for_spec(self.spec)), dst - ) - - # Finally re-target the symlinks that point to copied files. - for src, dst in delayed_links: - try: - s = os.stat(src) - target = copied_files[(s.st_dev, s.st_ino)] - except (OSError, KeyError): - target = None - if target: - os.symlink(os.path.relpath(target, os.path.dirname(dst)), dst) - else: - view.link(src, dst, spec=self.spec) - def test_hello_world(self): """run simple hello world program""" # do not use self.command because we are also testing the run env diff --git a/var/spack/repos/builtin/packages/qgis/package.py b/var/spack/repos/builtin/packages/qgis/package.py index 051a78660d..196a52622f 100644 --- a/var/spack/repos/builtin/packages/qgis/package.py +++ b/var/spack/repos/builtin/packages/qgis/package.py @@ -188,9 +188,7 @@ class Qgis(CMakePackage): @run_before("cmake", when="^py-pyqt5") def fix_pyqt5_cmake(self): cmfile = FileFilter(join_path("cmake", "FindPyQt5.cmake")) - pyqtpath = join_path( - self.spec["py-pyqt5"].prefix, self.spec["python"].package.platlib, "PyQt5" - ) + pyqtpath = join_path(self.spec["py-pyqt5"].package.module.python_platlib, "PyQt5") cmfile.filter( 'SET(PYQT5_MOD_DIR "${Python_SITEARCH}/PyQt5")', 'SET(PYQT5_MOD_DIR "' + pyqtpath + '")', @@ -210,7 +208,7 @@ class Qgis(CMakePackage): pyqtx = "PyQt6" sip_inc_dir = join_path( - self.spec["qscintilla"].prefix, self.spec["python"].package.platlib, pyqtx, "bindings" + self.spec["qscintilla"].package.module.python_platlib, pyqtx, "bindings" ) with open(join_path("python", "gui", "pyproject.toml.in"), "a") as tomlfile: tomlfile.write(f'\n[tool.sip.project]\nsip-include-dirs = ["{sip_inc_dir}"]\n') diff --git a/var/spack/repos/builtin/packages/qscintilla/package.py b/var/spack/repos/builtin/packages/qscintilla/package.py index bbf0b5a309..ce39d02d45 100644 --- a/var/spack/repos/builtin/packages/qscintilla/package.py +++ b/var/spack/repos/builtin/packages/qscintilla/package.py @@ -101,7 +101,7 @@ class Qscintilla(QMakePackage): with working_dir(join_path(self.stage.source_path, "Python")): copy(ftoml, "pyproject.toml") sip_inc_dir = join_path( - self.spec[py_pyqtx].prefix, self.spec["python"].package.platlib, pyqtx, "bindings" + self.spec[py_pyqtx].package.module.python_platlib, pyqtx, "bindings" ) with open("pyproject.toml", "a") as tomlfile: diff --git a/var/spack/repos/builtin/packages/redland-bindings/package.py b/var/spack/repos/builtin/packages/redland-bindings/package.py index 964f62108f..3c6bb42abc 100644 --- a/var/spack/repos/builtin/packages/redland-bindings/package.py +++ b/var/spack/repos/builtin/packages/redland-bindings/package.py @@ -3,8 +3,6 @@ # # SPDX-License-Identifier: (Apache-2.0 OR MIT) -import os - from spack.package import * @@ -27,11 +25,4 @@ class RedlandBindings(AutotoolsPackage): extends("python") def configure_args(self): - plib = self.spec["python"].prefix.lib - plib64 = self.spec["python"].prefix.lib64 - mybase = self.prefix.lib - if os.path.isdir(plib64) and not os.path.isdir(plib): - mybase = self.prefix.lib64 - pver = "python{0}".format(self.spec["python"].version.up_to(2)) - myplib = join_path(mybase, pver, "site-packages") - return ["--with-python", "PYTHON_LIB={0}".format(myplib)] + return ["--with-python", f"PYTHON_LIB={python_platlib}"] diff --git a/var/spack/repos/builtin/packages/z3/package.py b/var/spack/repos/builtin/packages/z3/package.py index e386e050c8..b42d46655a 100644 --- a/var/spack/repos/builtin/packages/z3/package.py +++ b/var/spack/repos/builtin/packages/z3/package.py @@ -53,13 +53,6 @@ class Z3(CMakePackage): ] if spec.satisfies("+python"): - args.append( - self.define( - "CMAKE_INSTALL_PYTHON_PKG_DIR", - join_path( - prefix.lib, "python%s" % spec["python"].version.up_to(2), "site-packages" - ), - ) - ) + args.append(self.define("CMAKE_INSTALL_PYTHON_PKG_DIR", python_platlib)) return args diff --git a/var/spack/repos/duplicates.test/packages/python-venv b/var/spack/repos/duplicates.test/packages/python-venv new file mode 120000 index 0000000000..a9a1ce867a --- /dev/null +++ b/var/spack/repos/duplicates.test/packages/python-venv @@ -0,0 +1 @@ +../../builtin.mock/packages/python-venv
\ No newline at end of file |