diff options
24 files changed, 34 insertions, 1708 deletions
diff --git a/lib/spack/docs/basic_usage.rst b/lib/spack/docs/basic_usage.rst index 73895449b0..b26429b248 100644 --- a/lib/spack/docs/basic_usage.rst +++ b/lib/spack/docs/basic_usage.rst @@ -1114,21 +1114,21 @@ set of arbitrary versions, such as ``@1.0,1.5,1.7`` (``1.0``, ``1.5``, or ``1.7``). When you supply such a specifier to ``spack install``, it constrains the set of versions that Spack will install. -For packages with a ``git`` attribute, ``git`` references -may be specified instead of a numerical version i.e. branches, tags -and commits. Spack will stage and build based off the ``git`` +For packages with a ``git`` attribute, ``git`` references +may be specified instead of a numerical version i.e. branches, tags +and commits. Spack will stage and build based off the ``git`` reference provided. Acceptable syntaxes for this are: .. code-block:: sh - + # branches and tags foo@git.develop # use the develop branch foo@git.0.19 # use the 0.19 tag - + # commit hashes foo@abcdef1234abcdef1234abcdef1234abcdef1234 # 40 character hashes are automatically treated as git commits foo@git.abcdef1234abcdef1234abcdef1234abcdef1234 - + Spack versions from git reference either have an associated version supplied by the user, or infer a relationship to known versions from the structure of the git repository. If an associated version is supplied by the user, Spack treats the git version as equivalent to that @@ -1745,13 +1745,13 @@ directly when you run ``python``: Using Extensions ^^^^^^^^^^^^^^^^ -There are four ways to get ``numpy`` working in Python. The first is +There are multiple ways to get ``numpy`` working in Python. The first is to use :ref:`shell-support`. You can simply ``load`` the extension, -and it will be added to the ``PYTHONPATH`` in your current shell: +and it will be added to the ``PYTHONPATH`` in your current shell, and +Python itself will be available in the ``PATH``: .. code-block:: console - $ spack load python $ spack load py-numpy Now ``import numpy`` will succeed for as long as you keep your current @@ -1777,128 +1777,13 @@ load, you can use the ``spack module tcl|lmod loads`` command to get the name of the module from the Spack spec. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Activating Extensions in a View +Extensions in an Environment ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Another way to use extensions is to create a view, which merges the python installation along with the extensions into a single prefix. -See :ref:`configuring_environment_views` for a more in-depth description -of views. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Activating Extensions Globally -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As an alternative to creating a merged prefix with Python and its extensions, -and prior to support for views, Spack has provided a means to install the -extension into the Spack installation prefix for the extendee. This has -typically been useful since extendable packages typically search their own -installation path for addons by default. - -Global activations are performed with the ``spack activate`` command: - -.. _cmd-spack-activate: - -^^^^^^^^^^^^^^^^^^ -``spack activate`` -^^^^^^^^^^^^^^^^^^ - -.. code-block:: console - - $ spack activate py-numpy - ==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=linux-debian7-x86_64-3c74eb69 for python@2.7.8%gcc@4.4.7. - ==> Activated extension py-nose@1.3.4%gcc@4.4.7 arch=linux-debian7-x86_64-5f70f816 for python@2.7.8%gcc@4.4.7. - ==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=linux-debian7-x86_64-66733244 for python@2.7.8%gcc@4.4.7. - -Several things have happened here. The user requested that -``py-numpy`` be activated in the ``python`` installation it was built -with. Spack knows that ``py-numpy`` depends on ``py-nose`` and -``py-setuptools``, so it activated those packages first. Finally, -once all dependencies were activated in the ``python`` installation, -``py-numpy`` was activated as well. - -If we run ``spack extensions`` again, we now see the three new -packages listed as activated: - -.. code-block:: console - - $ spack extensions python - ==> python@2.7.8%gcc@4.4.7 arch=linux-debian7-x86_64-703c7a96 - ==> 36 extensions: - geos py-ipython py-pexpect py-pyside py-sip - py-basemap py-libxml2 py-pil py-pytz py-six - py-biopython py-mako py-pmw py-rpy2 py-sympy - py-cython py-matplotlib py-pychecker py-scientificpython py-virtualenv - py-dateutil py-mpi4py py-pygments py-scikit-learn - py-epydoc py-mx py-pylint py-scipy - py-gnuplot py-nose py-pyparsing py-setuptools - py-h5py py-numpy py-pyqt py-shiboken - - ==> 12 installed: - -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- - py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2 - py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10 - py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1 - py-matplotlib@1.4.2 py-pyparsing@2.0.3 py-six@1.9.0 - - ==> 3 currently activated: - -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- - py-nose@1.3.4 py-numpy@1.9.1 py-setuptools@11.3.1 - -Now, when a user runs python, ``numpy`` will be available for import -*without* the user having to explicitly load it. ``python@2.7.8`` now -acts like a system Python installation with ``numpy`` installed inside -of it. - -Spack accomplishes this by symbolically linking the *entire* prefix of -the ``py-numpy`` package into the prefix of the ``python`` package. To the -python interpreter, it looks like ``numpy`` is installed in the -``site-packages`` directory. - -The only limitation of global activation is that you can only have a *single* -version of an extension activated at a time. This is because multiple -versions of the same extension would conflict if symbolically linked -into the same prefix. Users who want a different version of a package -can still get it by using environment modules or views, but they will have to -explicitly load their preferred version. - -^^^^^^^^^^^^^^^^^^^^^^^^^^ -``spack activate --force`` -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If, for some reason, you want to activate a package *without* its -dependencies, you can use ``spack activate --force``: - -.. code-block:: console - - $ spack activate --force py-numpy - ==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=linux-debian7-x86_64-66733244 for python@2.7.8%gcc@4.4.7. - -.. _cmd-spack-deactivate: - -^^^^^^^^^^^^^^^^^^^^ -``spack deactivate`` -^^^^^^^^^^^^^^^^^^^^ - -We've seen how activating an extension can be used to set up a default -version of a Python module. Obviously, you may want to change that at -some point. ``spack deactivate`` is the command for this. There are -several variants: - -* ``spack deactivate <extension>`` will deactivate a single - extension. If another activated extension depends on this one, - Spack will warn you and exit with an error. -* ``spack deactivate --force <extension>`` deactivates an extension - regardless of packages that depend on it. -* ``spack deactivate --all <extension>`` deactivates an extension and - all of its dependencies. Use ``--force`` to disregard dependents. -* ``spack deactivate --all <extendee>`` deactivates *all* activated - extensions of a package. For example, to deactivate *all* python - extensions, use: - - .. code-block:: console - - $ spack deactivate --all python +See :ref:`environments` for a more in-depth description +of environment views. ----------------------- Filesystem requirements diff --git a/lib/spack/docs/contribution_guide.rst b/lib/spack/docs/contribution_guide.rst index 837a850ff9..455a420ff2 100644 --- a/lib/spack/docs/contribution_guide.rst +++ b/lib/spack/docs/contribution_guide.rst @@ -253,27 +253,6 @@ to update them. multiple runs of ``spack style`` just to re-compute line numbers and makes it much easier to fix errors directly off of the CI output. -.. warning:: - - Flake8 and ``pep8-naming`` require a number of dependencies in order - to run. If you installed ``py-flake8`` and ``py-pep8-naming``, the - easiest way to ensure the right packages are on your ``PYTHONPATH`` is - to run:: - - spack activate py-flake8 - spack activate pep8-naming - - so that all of the dependencies are symlinked to a central - location. If you see an error message like: - - .. code-block:: console - - Traceback (most recent call last): - File: "/usr/bin/flake8", line 5, in <module> - from pkg_resources import load_entry_point - ImportError: No module named pkg_resources - - that means Flake8 couldn't find setuptools in your ``PYTHONPATH``. ^^^^^^^^^^^^^^^^^^^ Documentation Tests @@ -309,13 +288,9 @@ All of these can be installed with Spack, e.g. .. code-block:: console - $ spack activate py-sphinx - $ spack activate py-sphinx-rtd-theme - $ spack activate py-sphinxcontrib-programoutput + $ spack load py-sphinx py-sphinx-rtd-theme py-sphinxcontrib-programoutput - so that all of the dependencies are symlinked into that Python's - tree. Alternatively, you could arrange for their library - directories to be added to PYTHONPATH. If you see an error message + so that all of the dependencies are added to PYTHONPATH. If you see an error message like: .. code-block:: console diff --git a/lib/spack/docs/environments.rst b/lib/spack/docs/environments.rst index 9f75f789ae..2fd51baafb 100644 --- a/lib/spack/docs/environments.rst +++ b/lib/spack/docs/environments.rst @@ -233,8 +233,8 @@ packages will be listed as roots of the Environment. All of the Spack commands that act on the list of installed specs are Environment-sensitive in this way, including ``install``, -``uninstall``, ``activate``, ``deactivate``, ``find``, ``extensions``, -and more. In the :ref:`environment-configuration` section we will discuss +``uninstall``, ``find``, ``extensions``, and more. In the +:ref:`environment-configuration` section we will discuss Environment-sensitive commands further. ^^^^^^^^^^^^^^^^^^^^^ diff --git a/lib/spack/docs/packaging_guide.rst b/lib/spack/docs/packaging_guide.rst index 2ceb4ce051..fb1d5edcab 100644 --- a/lib/spack/docs/packaging_guide.rst +++ b/lib/spack/docs/packaging_guide.rst @@ -2722,67 +2722,6 @@ extensions; as a consequence python extension packages (those inheriting from ``PythonPackage``) likewise override ``add_files_to_view`` in order to rewrite shebang lines which point to the Python interpreter. -^^^^^^^^^^^^^^^^^^^^^^^^^ -Activation & deactivation -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Adding an extension to a view is referred to as an activation. If the view is -maintained in the Spack installation prefix of the extendee this is called a -global activation. Activations may involve updating some centralized state -that is maintained by the extendee package, so there can be additional work -for adding extensions compared with non-extension packages. - -Spack's ``Package`` class has default ``activate`` and ``deactivate`` -implementations that handle symbolically linking extensions' prefixes -into a specified view. Extendable packages can override these methods -to add custom activate/deactivate logic of their own. For example, -the ``activate`` and ``deactivate`` methods in the Python class handle -symbolic linking of extensions, but they also handle details surrounding -Python's ``.pth`` files, and other aspects of Python packaging. - -Spack's extensions mechanism is designed to be extensible, so that -other packages (like Ruby, R, Perl, etc.) can provide their own -custom extension management logic, as they may not handle modules the -same way that Python does. - -Let's look at Python's activate function: - -.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/python/package.py - :pyobject: Python.activate - :linenos: - -This function is called on the *extendee* (Python). It first calls -``activate`` in the superclass, which handles symlinking the -extension package's prefix into the specified view. It then does -some special handling of the ``easy-install.pth`` file, part of -Python's setuptools. - -Deactivate behaves similarly to activate, but it unlinks files: - -.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/python/package.py - :pyobject: Python.deactivate - :linenos: - -Both of these methods call some custom functions in the Python -package. See the source for Spack's Python package for details. - -^^^^^^^^^^^^^^^^^^^^ -Activation arguments -^^^^^^^^^^^^^^^^^^^^ - -You may have noticed that the ``activate`` function defined above -takes keyword arguments. These are the keyword arguments from -``extends()``, and they are passed to both activate and deactivate. - -This capability allows an extension to customize its own activation by -passing arguments to the extendee. Extendees can likewise implement -custom ``activate()`` and ``deactivate()`` functions to suit their -needs. - -The only keyword argument supported by default is the ``ignore`` -argument, which can take a regex, list of regexes, or a predicate to -determine which files *not* to symlink during activation. - .. _virtual-dependencies: -------------------- diff --git a/lib/spack/spack/cmd/activate.py b/lib/spack/spack/cmd/activate.py deleted file mode 100644 index bfb0de5528..0000000000 --- a/lib/spack/spack/cmd/activate.py +++ /dev/null @@ -1,53 +0,0 @@ -# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -import llnl.util.tty as tty - -import spack.cmd -import spack.cmd.common.arguments as arguments -import spack.environment as ev -from spack.filesystem_view import YamlFilesystemView - -description = "activate a package extension" -section = "extensions" -level = "long" - - -def setup_parser(subparser): - subparser.add_argument( - "-f", "--force", action="store_true", help="activate without first activating dependencies" - ) - subparser.add_argument("-v", "--view", metavar="VIEW", type=str, help="the view to operate on") - arguments.add_common_arguments(subparser, ["installed_spec"]) - - -def activate(parser, args): - - tty.warn( - "spack activate is deprecated in favor of " "environments and will be removed in v0.19.0" - ) - - specs = spack.cmd.parse_specs(args.spec) - if len(specs) != 1: - tty.die("activate requires one spec. %d given." % len(specs)) - - spec = spack.cmd.disambiguate_spec(specs[0], ev.active_environment()) - if not spec.package.is_extension: - tty.die("%s is not an extension." % spec.name) - - if args.view: - target = args.view - else: - target = spec.package.extendee_spec.prefix - - view = YamlFilesystemView(target, spack.store.layout) - - if spec.package.is_activated(view): - tty.msg("Package %s is already activated." % specs[0].short_spec) - return - - # TODO: refactor FilesystemView.add_extension and use that here (so there - # aren't two ways of activating extensions) - spec.package.do_activate(view, with_dependencies=not args.force) diff --git a/lib/spack/spack/cmd/deactivate.py b/lib/spack/spack/cmd/deactivate.py deleted file mode 100644 index 2fa18fc2b1..0000000000 --- a/lib/spack/spack/cmd/deactivate.py +++ /dev/null @@ -1,96 +0,0 @@ -# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -import llnl.util.tty as tty - -import spack.cmd -import spack.cmd.common.arguments as arguments -import spack.environment as ev -import spack.graph -import spack.store -from spack.filesystem_view import YamlFilesystemView - -description = "deactivate a package extension" -section = "extensions" -level = "long" - - -def setup_parser(subparser): - subparser.add_argument( - "-f", - "--force", - action="store_true", - help="run deactivation even if spec is NOT currently activated", - ) - subparser.add_argument("-v", "--view", metavar="VIEW", type=str, help="the view to operate on") - subparser.add_argument( - "-a", - "--all", - action="store_true", - help="deactivate all extensions of an extendable package, or " - "deactivate an extension AND its dependencies", - ) - arguments.add_common_arguments(subparser, ["installed_spec"]) - - -def deactivate(parser, args): - - tty.warn( - "spack deactivate is deprecated in favor of " "environments and will be removed in v0.19.0" - ) - - specs = spack.cmd.parse_specs(args.spec) - if len(specs) != 1: - tty.die("deactivate requires one spec. %d given." % len(specs)) - - env = ev.active_environment() - spec = spack.cmd.disambiguate_spec(specs[0], env) - pkg = spec.package - - if args.view: - target = args.view - elif pkg.is_extension: - target = pkg.extendee_spec.prefix - elif pkg.extendable: - target = spec.prefix - - view = YamlFilesystemView(target, spack.store.layout) - - if args.all: - if pkg.extendable: - tty.msg("Deactivating all extensions of %s" % pkg.spec.short_spec) - ext_pkgs = spack.store.db.activated_extensions_for(spec, view.extensions_layout) - - for ext_pkg in ext_pkgs: - ext_pkg.spec.normalize() - if ext_pkg.is_activated(view): - ext_pkg.do_deactivate(view, force=True) - - elif pkg.is_extension: - if not args.force and not spec.package.is_activated(view): - tty.die("%s is not activated." % pkg.spec.short_spec) - - tty.msg("Deactivating %s and all dependencies." % pkg.spec.short_spec) - - nodes_in_topological_order = spack.graph.topological_sort(spec) - for espec in reversed(nodes_in_topological_order): - epkg = espec.package - if epkg.extends(pkg.extendee_spec): - if epkg.is_activated(view) or args.force: - epkg.do_deactivate(view, force=args.force) - - else: - tty.die("spack deactivate --all requires an extendable package " "or an extension.") - - else: - if not pkg.is_extension: - tty.die( - "spack deactivate requires an extension.", "Did you mean 'spack deactivate --all'?" - ) - - if not args.force and not spec.package.is_activated(view): - tty.die("Package %s is not activated." % spec.short_spec) - - spec.package.do_deactivate(view, force=args.force) diff --git a/lib/spack/spack/cmd/extensions.py b/lib/spack/spack/cmd/extensions.py index fa79f8df2d..f169d0bc21 100644 --- a/lib/spack/spack/cmd/extensions.py +++ b/lib/spack/spack/cmd/extensions.py @@ -14,7 +14,6 @@ import spack.cmd.common.arguments as arguments import spack.environment as ev import spack.repo import spack.store -from spack.filesystem_view import YamlFilesystemView description = "list extensions for package" section = "extensions" @@ -38,10 +37,9 @@ def setup_parser(subparser): "--show", action="store", default="all", - choices=("packages", "installed", "activated", "all"), + choices=("packages", "installed", "all"), help="show only part of output", ) - subparser.add_argument("-v", "--view", metavar="VIEW", type=str, help="the view to operate on") subparser.add_argument( "spec", @@ -91,13 +89,6 @@ def extensions(parser, args): tty.msg("%d extensions:" % len(extensions)) colify(ext.name for ext in extensions) - if args.view: - target = args.view - else: - target = spec.prefix - - view = YamlFilesystemView(target, spack.store.layout) - if args.show in ("installed", "all"): # List specs of installed extensions. installed = [s.spec for s in spack.store.db.installed_extensions_for(spec)] @@ -109,14 +100,3 @@ def extensions(parser, args): else: tty.msg("%d installed:" % len(installed)) cmd.display_specs(installed, args) - - if args.show in ("activated", "all"): - # List specs of activated extensions. - activated = view.extensions_layout.extension_map(spec) - if args.show == "all": - print - if not activated: - tty.msg("None activated.") - else: - tty.msg("%d activated:" % len(activated)) - cmd.display_specs(activated.values(), args) diff --git a/lib/spack/spack/database.py b/lib/spack/spack/database.py index 30ec4d8da0..74e2ec98c3 100644 --- a/lib/spack/spack/database.py +++ b/lib/spack/spack/database.py @@ -53,7 +53,6 @@ from spack.directory_layout import ( InconsistentInstallDirectoryError, ) from spack.error import SpackError -from spack.filesystem_view import YamlFilesystemView from spack.util.crypto import bit_length from spack.version import Version @@ -1379,23 +1378,6 @@ class Database(object): if spec.package.extends(extendee_spec): yield spec.package - @_autospec - def activated_extensions_for(self, extendee_spec, extensions_layout=None): - """ - Return the specs of all packages that extend - the given spec - """ - if extensions_layout is None: - view = YamlFilesystemView(extendee_spec.prefix, spack.store.layout) - extensions_layout = view.extensions_layout - for spec in self.query(): - try: - extensions_layout.check_activated(extendee_spec, spec) - yield spec.package - except spack.directory_layout.NoSuchExtensionError: - continue - # TODO: conditional way to do this instead of catching exceptions - def _get_by_hash_local(self, dag_hash, default=None, installed=any): # hash is a full hash and is in the data somewhere if dag_hash in self._data: diff --git a/lib/spack/spack/directives.py b/lib/spack/spack/directives.py index 7c49158c79..a4c13c70b7 100644 --- a/lib/spack/spack/directives.py +++ b/lib/spack/spack/directives.py @@ -468,14 +468,7 @@ def depends_on(spec, when=None, type=default_deptype, patches=None): @directive(("extendees", "dependencies")) def extends(spec, type=("build", "run"), **kwargs): - """Same as depends_on, but allows symlinking into dependency's - prefix tree. - - This is for Python and other language modules where the module - needs to be installed into the prefix of the Python installation. - Spack handles this by installing modules into their own prefix, - but allowing ONE module version to be symlinked into a parent - Python install at a time, using ``spack activate``. + """Same as depends_on, but also adds this package to the extendee list. keyword arguments can be passed to extends() so that extension packages can pass parameters to the extendee's extension diff --git a/lib/spack/spack/directory_layout.py b/lib/spack/spack/directory_layout.py index b5848f12a7..607f8bc39d 100644 --- a/lib/spack/spack/directory_layout.py +++ b/lib/spack/spack/directory_layout.py @@ -10,10 +10,8 @@ import posixpath import re import shutil import sys -import tempfile from contextlib import contextmanager -import ruamel.yaml as yaml import six import llnl.util.filesystem as fs @@ -389,205 +387,6 @@ class DirectoryLayout(object): path = os.path.dirname(path) -class ExtensionsLayout(object): - """A directory layout is used to associate unique paths with specs for - package extensions. - Keeps track of which extensions are activated for what package. - Depending on the use case, this can mean globally activated extensions - directly in the installation folder - or extensions activated in - filesystem views. - """ - - def __init__(self, view, **kwargs): - self.view = view - - def add_extension(self, spec, ext_spec): - """Add to the list of currently installed extensions.""" - raise NotImplementedError() - - def check_activated(self, spec, ext_spec): - """Ensure that ext_spec can be removed from spec. - - If not, raise NoSuchExtensionError. - """ - raise NotImplementedError() - - def check_extension_conflict(self, spec, ext_spec): - """Ensure that ext_spec can be activated in spec. - - If not, raise ExtensionAlreadyInstalledError or - ExtensionConflictError. - """ - raise NotImplementedError() - - def extension_map(self, spec): - """Get a dict of currently installed extension packages for a spec. - - Dict maps { name : extension_spec } - Modifying dict does not affect internals of this layout. - """ - raise NotImplementedError() - - def extendee_target_directory(self, extendee): - """Specify to which full path extendee should link all files - from extensions.""" - raise NotImplementedError - - def remove_extension(self, spec, ext_spec): - """Remove from the list of currently installed extensions.""" - raise NotImplementedError() - - -class YamlViewExtensionsLayout(ExtensionsLayout): - """Maintain extensions within a view.""" - - def __init__(self, view, layout): - """layout is the corresponding YamlDirectoryLayout object for which - we implement extensions. - """ - super(YamlViewExtensionsLayout, self).__init__(view) - self.layout = layout - self.extension_file_name = "extensions.yaml" - - # Cache of already written/read extension maps. - self._extension_maps = {} - - def add_extension(self, spec, ext_spec): - _check_concrete(spec) - _check_concrete(ext_spec) - - # Check whether it's already installed or if it's a conflict. - exts = self._extension_map(spec) - self.check_extension_conflict(spec, ext_spec) - - # do the actual adding. - exts[ext_spec.name] = ext_spec - self._write_extensions(spec, exts) - - def check_extension_conflict(self, spec, ext_spec): - exts = self._extension_map(spec) - if ext_spec.name in exts: - installed_spec = exts[ext_spec.name] - if ext_spec.dag_hash() == installed_spec.dag_hash(): - raise ExtensionAlreadyInstalledError(spec, ext_spec) - else: - raise ExtensionConflictError(spec, ext_spec, installed_spec) - - def check_activated(self, spec, ext_spec): - exts = self._extension_map(spec) - if (ext_spec.name not in exts) or (ext_spec != exts[ext_spec.name]): - raise NoSuchExtensionError(spec, ext_spec) - - def extension_file_path(self, spec): - """Gets full path to an installed package's extension file, which - keeps track of all the extensions for that package which have been - added to this view. - """ - _check_concrete(spec) - normalize_path = lambda p: (os.path.abspath(p).rstrip(os.path.sep)) - - view_prefix = self.view.get_projection_for_spec(spec) - if normalize_path(spec.prefix) == normalize_path(view_prefix): - # For backwards compatibility, when the view is the extended - # package's installation directory, do not include the spec name - # as a subdirectory. - components = [view_prefix, self.layout.metadata_dir, self.extension_file_name] - else: - components = [ - view_prefix, - self.layout.metadata_dir, - spec.name, - self.extension_file_name, - ] - - return os.path.join(*components) - - def extension_map(self, spec): - """Defensive copying version of _extension_map() for external API.""" - _check_concrete(spec) - return self._extension_map(spec).copy() - - def remove_extension(self, spec, ext_spec): - _check_concrete(spec) - _check_concrete(ext_spec) - - # Make sure it's installed before removing. - exts = self._extension_map(spec) - self.check_activated(spec, ext_spec) - - # do the actual removing. - del exts[ext_spec.name] - self._write_extensions(spec, exts) - - def _extension_map(self, spec): - """Get a dict<name -> spec> for all extensions currently - installed for this package.""" - _check_concrete(spec) - - if spec not in self._extension_maps: - path = self.extension_file_path(spec) - if not os.path.exists(path): - self._extension_maps[spec] = {} - - else: - by_hash = self.layout.specs_by_hash() - exts = {} - with open(path) as ext_file: - yaml_file = yaml.load(ext_file) - for entry in yaml_file["extensions"]: - name = next(iter(entry)) - dag_hash = entry[name]["hash"] - prefix = entry[name]["path"] - - if dag_hash not in by_hash: - raise InvalidExtensionSpecError( - "Spec %s not found in %s" % (dag_hash, prefix) - ) - - ext_spec = by_hash[dag_hash] - if prefix != ext_spec.prefix: - raise InvalidExtensionSpecError( - "Prefix %s does not match spec hash %s: %s" - % (prefix, dag_hash, ext_spec) - ) - - exts[ext_spec.name] = ext_spec - self._extension_maps[spec] = exts - - return self._extension_maps[spec] - - def _write_extensions(self, spec, extensions): - path = self.extension_file_path(spec) - - if not extensions: - # Remove the empty extensions file - os.remove(path) - return - - # Create a temp file in the same directory as the actual file. - dirname, basename = os.path.split(path) - fs.mkdirp(dirname) - - tmp = tempfile.NamedTemporaryFile(prefix=basename, dir=dirname, delete=False) - - # write tmp file - with tmp: - yaml.dump( - { - "extensions": [ - {ext.name: {"hash": ext.dag_hash(), "path": str(ext.prefix)}} - for ext in sorted(extensions.values()) - ] - }, - tmp, - default_flow_style=False, - encoding="utf-8", - ) - - # Atomic update by moving tmpfile on top of old one. - fs.rename(tmp.name, path) - - class DirectoryLayoutError(SpackError): """Superclass for directory layout errors.""" @@ -644,13 +443,3 @@ class ExtensionConflictError(DirectoryLayoutError): "%s cannot be installed in %s because it conflicts with %s" % (ext_spec.short_spec, spec.short_spec, conflict.short_spec) ) - - -class NoSuchExtensionError(DirectoryLayoutError): - """Raised when an extension isn't there on deactivate.""" - - def __init__(self, spec, ext_spec): - super(NoSuchExtensionError, self).__init__( - "%s cannot be removed from %s because it's not activated." - % (ext_spec.short_spec, spec.short_spec) - ) diff --git a/lib/spack/spack/filesystem_view.py b/lib/spack/spack/filesystem_view.py index db2d6c9477..f8cc382ef7 100644 --- a/lib/spack/spack/filesystem_view.py +++ b/lib/spack/spack/filesystem_view.py @@ -37,10 +37,6 @@ import spack.spec import spack.store import spack.util.spack_json as s_json import spack.util.spack_yaml as s_yaml -from spack.directory_layout import ( - ExtensionAlreadyInstalledError, - YamlViewExtensionsLayout, -) from spack.error import SpackError __all__ = ["FilesystemView", "YamlFilesystemView"] @@ -166,9 +162,6 @@ class FilesystemView(object): """ Add given specs to view. - The supplied specs might be standalone packages or extensions of - other packages. - Should accept `with_dependencies` as keyword argument (default True) to indicate wether or not dependencies should be activated as well. @@ -176,13 +169,7 @@ class FilesystemView(object): Should except an `exclude` keyword argument containing a list of regexps that filter out matching spec names. - This method should make use of `activate_{extension,standalone}`. - """ - raise NotImplementedError - - def add_extension(self, spec): - """ - Add (link) an extension in this view. Does not add dependencies. + This method should make use of `activate_standalone`. """ raise NotImplementedError @@ -202,9 +189,6 @@ class FilesystemView(object): """ Removes given specs from view. - The supplied spec might be a standalone package or an extension of - another package. - Should accept `with_dependencies` as keyword argument (default True) to indicate wether or not dependencies should be deactivated as well. @@ -216,13 +200,7 @@ class FilesystemView(object): Should except an `exclude` keyword argument containing a list of regexps that filter out matching spec names. - This method should make use of `deactivate_{extension,standalone}`. - """ - raise NotImplementedError - - def remove_extension(self, spec): - """ - Remove (unlink) an extension from this view. + This method should make use of `deactivate_standalone`. """ raise NotImplementedError @@ -296,8 +274,6 @@ class YamlFilesystemView(FilesystemView): msg += " which does not match projections passed manually." raise ConflictingProjectionsError(msg) - self.extensions_layout = YamlViewExtensionsLayout(self, layout) - self._croot = colorize_root(self._root) + " " def write_projections(self): @@ -332,38 +308,10 @@ class YamlFilesystemView(FilesystemView): self.print_conflict(v, s) return - extensions = set(filter(lambda s: s.package.is_extension, specs)) - standalones = specs - extensions - - set(map(self._check_no_ext_conflicts, extensions)) - # fail on first error, otherwise link extensions as well - if all(map(self.add_standalone, standalones)): - all(map(self.add_extension, extensions)) - - def add_extension(self, spec): - if not spec.package.is_extension: - tty.error(self._croot + "Package %s is not an extension." % spec.name) - return False - - if spec.external: - tty.warn(self._croot + "Skipping external package: %s" % colorize_spec(spec)) - return True - - if not spec.package.is_activated(self): - spec.package.do_activate(self, verbose=self.verbose, with_dependencies=False) - - # make sure the meta folder is linked as well (this is not done by the - # extension-activation mechnism) - if not self.check_added(spec): - self.link_meta_folder(spec) - - return True + for s in specs: + self.add_standalone(s) def add_standalone(self, spec): - if spec.package.is_extension: - tty.error(self._croot + "Package %s is an extension." % spec.name) - return False - if spec.external: tty.warn(self._croot + "Skipping external package: %s" % colorize_spec(spec)) return True @@ -372,19 +320,6 @@ class YamlFilesystemView(FilesystemView): tty.warn(self._croot + "Skipping already linked package: %s" % colorize_spec(spec)) return True - if spec.package.extendable: - # Check for globally activated extensions in the extendee that - # we're looking at. - activated = [p.spec for p in spack.store.db.activated_extensions_for(spec)] - if activated: - tty.error( - "Globally activated extensions cannot be used in " - "conjunction with filesystem views. " - "Please deactivate the following specs: " - ) - spack.cmd.display_specs(activated, flags=True, variants=True, long=False) - return False - self.merge(spec) self.link_meta_folder(spec) @@ -533,27 +468,10 @@ class YamlFilesystemView(FilesystemView): # Remove the packages from the view for spec in to_deactivate_sorted: - if spec.package.is_extension: - self.remove_extension(spec, with_dependents=with_dependents) - else: - self.remove_standalone(spec) + self.remove_standalone(spec) self._purge_empty_directories() - def remove_extension(self, spec, with_dependents=True): - """ - Remove (unlink) an extension from this view. - """ - if not self.check_added(spec): - tty.warn(self._croot + "Skipping package not linked in view: %s" % spec.name) - return - - if spec.package.is_activated(self): - spec.package.do_deactivate( - self, verbose=self.verbose, remove_dependents=with_dependents - ) - self.unlink_meta_folder(spec) - def remove_standalone(self, spec): """ Remove (unlink) a standalone package from this view. @@ -575,14 +493,9 @@ class YamlFilesystemView(FilesystemView): Relies on the ordering of projections to avoid ambiguity. """ spec = spack.spec.Spec(spec) - # Extensions are placed by their extendee, not by their own spec - locator_spec = spec - if spec.package.extendee_spec: - locator_spec = spec.package.extendee_spec - - proj = spack.projections.get_projection(self.projections, locator_spec) + proj = spack.projections.get_projection(self.projections, spec) if proj: - return os.path.join(self._root, locator_spec.format(proj)) + return os.path.join(self._root, spec.format(proj)) return self._root def get_all_specs(self): @@ -712,18 +625,6 @@ class YamlFilesystemView(FilesystemView): assert os.path.exists(path) shutil.rmtree(path) - def _check_no_ext_conflicts(self, spec): - """ - Check that there is no extension conflict for specs. - """ - extendee = spec.package.extendee_spec - try: - self.extensions_layout.check_extension_conflict(extendee, spec) - except ExtensionAlreadyInstalledError: - # we print the warning here because later on the order in which - # packages get activated is not clear (set-sorting) - tty.warn(self._croot + "Skipping already activated package: %s" % spec.name) - class SimpleFilesystemView(FilesystemView): """A simple and partial implementation of FilesystemView focused on @@ -842,14 +743,9 @@ class SimpleFilesystemView(FilesystemView): Relies on the ordering of projections to avoid ambiguity. """ spec = spack.spec.Spec(spec) - # Extensions are placed by their extendee, not by their own spec - locator_spec = spec - if spec.package.extendee_spec: - locator_spec = spec.package.extendee_spec - - proj = spack.projections.get_projection(self.projections, locator_spec) + proj = spack.projections.get_projection(self.projections, spec) if proj: - return os.path.join(self._root, locator_spec.format(proj)) + return os.path.join(self._root, spec.format(proj)) return self._root diff --git a/lib/spack/spack/hooks/extensions.py b/lib/spack/spack/hooks/extensions.py deleted file mode 100644 index 56d7c04550..0000000000 --- a/lib/spack/spack/hooks/extensions.py +++ /dev/null @@ -1,20 +0,0 @@ -# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -import spack -from spack.filesystem_view import YamlFilesystemView - - -def pre_uninstall(spec): - pkg = spec.package - assert spec.concrete - - if pkg.is_extension: - target = pkg.extendee_spec.prefix - view = YamlFilesystemView(target, spack.store.layout) - - if pkg.is_activated(view): - # deactivate globally - pkg.do_deactivate(force=True) diff --git a/lib/spack/spack/package_base.py b/lib/spack/spack/package_base.py index 48b036e8cf..4e25cb6b04 100644 --- a/lib/spack/spack/package_base.py +++ b/lib/spack/spack/package_base.py @@ -1313,19 +1313,6 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM s = self.extendee_spec return s and spec.satisfies(s) - def is_activated(self, view): - """Return True if package is activated.""" - if not self.is_extension: - raise ValueError("is_activated called on package that is not an extension.") - if self.extendee_spec.installed_upstream: - # If this extends an upstream package, it cannot be activated for - # it. This bypasses construction of the extension map, which can - # can fail when run in the context of a downstream Spack instance - return False - extensions_layout = view.extensions_layout - exts = extensions_layout.extension_map(self.extendee_spec) - return (self.name in exts) and (exts[self.name] == self.spec) - def provides(self, vpkg_name): """ True if this package provides a virtual package with the specified name @@ -2325,30 +2312,6 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM """Deprecate this package in favor of deprecator spec""" spec = self.spec - # Check whether package to deprecate has active extensions - if self.extendable: - view = spack.filesystem_view.YamlFilesystemView(spec.prefix, spack.store.layout) - active_exts = view.extensions_layout.extension_map(spec).values() - if active_exts: - short = spec.format("{name}/{hash:7}") - m = "Spec %s has active extensions\n" % short - for active in active_exts: - m += " %s\n" % active.format("{name}/{hash:7}") - m += "Deactivate extensions before deprecating %s" % short - tty.die(m) - - # Check whether package to deprecate is an active extension - if self.is_extension: - extendee = self.extendee_spec - view = spack.filesystem_view.YamlFilesystemView(extendee.prefix, spack.store.layout) - - if self.is_activated(view): - short = spec.format("{name}/{hash:7}") - short_ext = extendee.format("{name}/{hash:7}") - msg = "Spec %s is an active extension of %s\n" % (short, short_ext) - msg += "Deactivate %s to be able to deprecate it" % short - tty.die(msg) - # Install deprecator if it isn't installed already if not spack.store.db.query(deprecator): deprecator.package.do_install() @@ -2378,155 +2341,6 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM if not self.extendable: raise ValueError("Package %s is not extendable!" % self.name) - def _sanity_check_extension(self): - if not self.is_extension: - raise ActivationError("This package is not an extension.") - - extendee_package = self.extendee_spec.package - extendee_package._check_extendable() - - if not self.extendee_spec.installed: - raise ActivationError("Can only (de)activate extensions for installed packages.") - if not self.spec.installed: - raise ActivationError("Extensions must first be installed.") - if self.extendee_spec.name not in self.extendees: - raise ActivationError("%s does not extend %s!" % (self.name, self.extendee.name)) - - def do_activate(self, view=None, with_dependencies=True, verbose=True): - """Called on an extension to invoke the extendee's activate method. - - Commands should call this routine, and should not call - activate() directly. - """ - if verbose: - tty.msg( - "Activating extension {0} for {1}".format( - self.spec.cshort_spec, self.extendee_spec.cshort_spec - ) - ) - - self._sanity_check_extension() - if not view: - view = YamlFilesystemView(self.extendee_spec.prefix, spack.store.layout) - - extensions_layout = view.extensions_layout - - try: - extensions_layout.check_extension_conflict(self.extendee_spec, self.spec) - except spack.directory_layout.ExtensionAlreadyInstalledError as e: - # already installed, let caller know - tty.msg(e.message) - return - - # Activate any package dependencies that are also extensions. - if with_dependencies: - for spec in self.dependency_activations(): - if not spec.package.is_activated(view): - spec.package.do_activate( - view, with_dependencies=with_dependencies, verbose=verbose - ) - - self.extendee_spec.package.activate(self, view, **self.extendee_args) - - extensions_layout.add_extension(self.extendee_spec, self.spec) - - if verbose: - tty.debug( - "Activated extension {0} for {1}".format( - self.spec.cshort_spec, self.extendee_spec.cshort_spec - ) - ) - - def dependency_activations(self): - return ( - spec - for spec in self.spec.traverse(root=False, deptype="run") - if spec.package.extends(self.extendee_spec) - ) - - def activate(self, extension, view, **kwargs): - """ - Add the extension to the specified view. - - Package authors can override this function to maintain some - centralized state related to the set of activated extensions - for a package. - - Spack internals (commands, hooks, etc.) should call - do_activate() method so that proper checks are always executed. - """ - view.merge(extension.spec, ignore=kwargs.get("ignore", None)) - - def do_deactivate(self, view=None, **kwargs): - """Remove this extension package from the specified view. Called - on the extension to invoke extendee's deactivate() method. - - `remove_dependents=True` deactivates extensions depending on this - package instead of raising an error. - """ - self._sanity_check_extension() - force = kwargs.get("force", False) - verbose = kwargs.get("verbose", True) - remove_dependents = kwargs.get("remove_dependents", False) - - if verbose: - tty.msg( - "Deactivating extension {0} for {1}".format( - self.spec.cshort_spec, self.extendee_spec.cshort_spec - ) - ) - - if not view: - view = YamlFilesystemView(self.extendee_spec.prefix, spack.store.layout) - extensions_layout = view.extensions_layout - - # Allow a force deactivate to happen. This can unlink - # spurious files if something was corrupted. - if not force: - extensions_layout.check_activated(self.extendee_spec, self.spec) - - activated = extensions_layout.extension_map(self.extendee_spec) - for name, aspec in activated.items(): - if aspec == self.spec: - continue - for dep in aspec.traverse(deptype="run"): - if self.spec == dep: - if remove_dependents: - aspec.package.do_deactivate(**kwargs) - else: - msg = ( - "Cannot deactivate {0} because {1} is " - "activated and depends on it" - ) - raise ActivationError( - msg.format(self.spec.cshort_spec, aspec.cshort_spec) - ) - - self.extendee_spec.package.deactivate(self, view, **self.extendee_args) - - # redundant activation check -- makes SURE the spec is not - # still activated even if something was wrong above. - if self.is_activated(view): - extensions_layout.remove_extension(self.extendee_spec, self.spec) - - if verbose: - tty.debug( - "Deactivated extension {0} for {1}".format( - self.spec.cshort_spec, self.extendee_spec.cshort_spec - ) - ) - - def deactivate(self, extension, view, **kwargs): - """ - Remove all extension files from the specified view. - - Package authors can override this method to support other - extension mechanisms. Spack internals (commands, hooks, etc.) - should call do_deactivate() method so that proper checks are - always executed. - """ - view.unmerge(extension.spec, ignore=kwargs.get("ignore", None)) - def view(self): """Create a view with the prefix of this package as the root. Extensions added to this view will modify the installation prefix of diff --git a/lib/spack/spack/test/cmd/activate.py b/lib/spack/spack/test/cmd/activate.py deleted file mode 100644 index 9ba35cc988..0000000000 --- a/lib/spack/spack/test/cmd/activate.py +++ /dev/null @@ -1,41 +0,0 @@ -# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) -import sys - -import pytest - -from spack.main import SpackCommand - -activate = SpackCommand("activate") -deactivate = SpackCommand("deactivate") -install = SpackCommand("install") -extensions = SpackCommand("extensions") - -pytestmark = pytest.mark.skipif(sys.platform == "win32", reason="does not run on windows") - - -def test_activate(mock_packages, mock_archive, mock_fetch, config, install_mockery): - install("extension1") - activate("extension1") - output = extensions("--show", "activated", "extendee") - assert "extension1" in output - - -def test_deactivate(mock_packages, mock_archive, mock_fetch, config, install_mockery): - install("extension1") - activate("extension1") - deactivate("extension1") - output = extensions("--show", "activated", "extendee") - assert "extension1" not in output - - -def test_deactivate_all(mock_packages, mock_archive, mock_fetch, config, install_mockery): - install("extension1") - install("extension2") - activate("extension1") - activate("extension2") - deactivate("--all", "extendee") - output = extensions("--show", "activated", "extendee") - assert "extension1" not in output diff --git a/lib/spack/spack/test/cmd/deprecate.py b/lib/spack/spack/test/cmd/deprecate.py index b1fe8ca059..105dadab30 100644 --- a/lib/spack/spack/test/cmd/deprecate.py +++ b/lib/spack/spack/test/cmd/deprecate.py @@ -15,7 +15,6 @@ install = SpackCommand("install") uninstall = SpackCommand("uninstall") deprecate = SpackCommand("deprecate") find = SpackCommand("find") -activate = SpackCommand("activate") pytestmark = pytest.mark.skipif(sys.platform == "win32", reason="does not run on windows") @@ -89,24 +88,6 @@ def test_deprecate_deps(mock_packages, mock_archive, mock_fetch, install_mockery assert sorted(deprecated) == sorted(list(old_spec.traverse())) -def test_deprecate_fails_active_extensions( - mock_packages, mock_archive, mock_fetch, install_mockery -): - """Tests that active extensions and their extendees cannot be - deprecated.""" - install("extendee") - install("extension1") - activate("extension1") - - output = deprecate("-yi", "extendee", "extendee@nonexistent", fail_on_error=False) - assert "extension1" in output - assert "Deactivate extensions before deprecating" in output - - output = deprecate("-yiD", "extension1", "extension1@notaversion", fail_on_error=False) - assert "extendee" in output - assert "is an active extension of" in output - - def test_uninstall_deprecated(mock_packages, mock_archive, mock_fetch, install_mockery): """Tests that we can still uninstall deprecated packages.""" install("libelf@0.8.13") diff --git a/lib/spack/spack/test/cmd/extensions.py b/lib/spack/spack/test/cmd/extensions.py index c4e849365f..1ccda8aef1 100644 --- a/lib/spack/spack/test/cmd/extensions.py +++ b/lib/spack/spack/test/cmd/extensions.py @@ -35,12 +35,11 @@ def python_database(mock_packages, mutable_database): def test_extensions(mock_packages, python_database, config, capsys): ext2 = Spec("py-extension2").concretized() - def check_output(ni, na): + def check_output(ni): with capsys.disabled(): output = extensions("python") packages = extensions("-s", "packages", "python") installed = extensions("-s", "installed", "python") - activated = extensions("-s", "activated", "python") assert "==> python@2.7.11" in output assert "==> 2 extensions" in output assert "py-extension1" in output @@ -50,26 +49,13 @@ def test_extensions(mock_packages, python_database, config, capsys): assert "py-extension1" in packages assert "py-extension2" in packages assert "installed" not in packages - assert "activated" not in packages assert ("%s installed" % (ni if ni else "None")) in output - assert ("%s activated" % (na if na else "None")) in output assert ("%s installed" % (ni if ni else "None")) in installed - assert ("%s activated" % (na if na else "None")) in activated - - check_output(2, 0) - - ext2.package.do_activate() - check_output(2, 2) - - ext2.package.do_deactivate(force=True) - check_output(2, 1) - - ext2.package.do_activate() - check_output(2, 2) + check_output(2) ext2.package.do_uninstall(force=True) - check_output(1, 1) + check_output(1) def test_extensions_no_arguments(mock_packages): diff --git a/lib/spack/spack/test/cmd/view.py b/lib/spack/spack/test/cmd/view.py index 67c4275ddd..84693f1bf4 100644 --- a/lib/spack/spack/test/cmd/view.py +++ b/lib/spack/spack/test/cmd/view.py @@ -12,7 +12,6 @@ import spack.util.spack_yaml as s_yaml from spack.main import SpackCommand from spack.spec import Spec -activate = SpackCommand("activate") extensions = SpackCommand("extensions") install = SpackCommand("install") view = SpackCommand("view") @@ -135,46 +134,9 @@ def test_view_extension(tmpdir, mock_packages, mock_archive, mock_fetch, config, assert "extension1@1.0" in all_installed assert "extension1@2.0" in all_installed assert "extension2@1.0" in all_installed - global_activated = extensions("--show", "activated", "extendee") - assert "extension1@1.0" not in global_activated - assert "extension1@2.0" not in global_activated - assert "extension2@1.0" not in global_activated - view_activated = extensions("--show", "activated", "-v", viewpath, "extendee") - assert "extension1@1.0" in view_activated - assert "extension1@2.0" not in view_activated - assert "extension2@1.0" not in view_activated assert os.path.exists(os.path.join(viewpath, "bin", "extension1")) -def test_view_extension_projection( - tmpdir, mock_packages, mock_archive, mock_fetch, config, install_mockery -): - install("extendee@1.0") - install("extension1@1.0") - install("extension1@2.0") - install("extension2@1.0") - - viewpath = str(tmpdir.mkdir("view")) - view_projection = {"all": "{name}-{version}"} - projection_file = create_projection_file(tmpdir, view_projection) - view("symlink", viewpath, "--projection-file={0}".format(projection_file), "extension1@1.0") - - all_installed = extensions("--show", "installed", "extendee") - assert "extension1@1.0" in all_installed - assert "extension1@2.0" in all_installed - assert "extension2@1.0" in all_installed - global_activated = extensions("--show", "activated", "extendee") - assert "extension1@1.0" not in global_activated - assert "extension1@2.0" not in global_activated - assert "extension2@1.0" not in global_activated - view_activated = extensions("--show", "activated", "-v", viewpath, "extendee") - assert "extension1@1.0" in view_activated - assert "extension1@2.0" not in view_activated - assert "extension2@1.0" not in view_activated - - assert os.path.exists(os.path.join(viewpath, "extendee-1.0", "bin", "extension1")) - - def test_view_extension_remove( tmpdir, mock_packages, mock_archive, mock_fetch, config, install_mockery ): @@ -185,10 +147,6 @@ def test_view_extension_remove( view("remove", viewpath, "extension1@1.0") all_installed = extensions("--show", "installed", "extendee") assert "extension1@1.0" in all_installed - global_activated = extensions("--show", "activated", "extendee") - assert "extension1@1.0" not in global_activated - view_activated = extensions("--show", "activated", "-v", viewpath, "extendee") - assert "extension1@1.0" not in view_activated assert not os.path.exists(os.path.join(viewpath, "bin", "extension1")) @@ -217,46 +175,6 @@ def test_view_extension_conflict_ignored( assert fin.read() == "1.0" -def test_view_extension_global_activation( - tmpdir, mock_packages, mock_archive, mock_fetch, config, install_mockery -): - install("extendee") - install("extension1@1.0") - install("extension1@2.0") - install("extension2@1.0") - viewpath = str(tmpdir.mkdir("view")) - view("symlink", viewpath, "extension1@1.0") - activate("extension1@2.0") - activate("extension2@1.0") - all_installed = extensions("--show", "installed", "extendee") - assert "extension1@1.0" in all_installed - assert "extension1@2.0" in all_installed - assert "extension2@1.0" in all_installed - global_activated = extensions("--show", "activated", "extendee") - assert "extension1@1.0" not in global_activated - assert "extension1@2.0" in global_activated - assert "extension2@1.0" in global_activated - view_activated = extensions("--show", "activated", "-v", viewpath, "extendee") - assert "extension1@1.0" in view_activated - assert "extension1@2.0" not in view_activated - assert "extension2@1.0" not in view_activated - assert os.path.exists(os.path.join(viewpath, "bin", "extension1")) - assert not os.path.exists(os.path.join(viewpath, "bin", "extension2")) - - -def test_view_extendee_with_global_activations( - tmpdir, mock_packages, mock_archive, mock_fetch, config, install_mockery -): - install("extendee") - install("extension1@1.0") - install("extension1@2.0") - install("extension2@1.0") - viewpath = str(tmpdir.mkdir("view")) - activate("extension1@2.0") - output = view("symlink", viewpath, "extension1@1.0") - assert "Error: Globally activated extensions cannot be used" in output - - def test_view_fails_with_missing_projections_file(tmpdir): viewpath = str(tmpdir.mkdir("view")) projection_file = os.path.join(str(tmpdir), "nonexistent") diff --git a/lib/spack/spack/test/packages.py b/lib/spack/spack/test/packages.py index 3a22abcd7c..8b13c13fbb 100644 --- a/lib/spack/spack/test/packages.py +++ b/lib/spack/spack/test/packages.py @@ -84,12 +84,6 @@ class TestPackage(object): # Will error if inheritor package cannot find inherited patch files s.concretize() - def test_dependency_extensions(self): - s = Spec("extension2") - s.concretize() - deps = set(x.name for x in s.package.dependency_activations()) - assert deps == set(["extension1"]) - def test_import_class_from_package(self): from spack.pkg.builtin.mock.mpich import Mpich # noqa: F401 diff --git a/lib/spack/spack/test/test_activations.py b/lib/spack/spack/test/test_activations.py deleted file mode 100644 index b9a478b5d3..0000000000 --- a/lib/spack/spack/test/test_activations.py +++ /dev/null @@ -1,402 +0,0 @@ -# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -"""This includes tests for customized activation logic for specific packages - (e.g. python and perl). -""" - -import os -import sys - -import pytest - -from llnl.util.link_tree import MergeConflictError - -import spack.package_base -import spack.spec -from spack.directory_layout import DirectoryLayout -from spack.filesystem_view import YamlFilesystemView - -pytestmark = pytest.mark.skipif( - sys.platform == "win32", - reason="Python activation not currently supported on Windows", -) - - -def create_ext_pkg(name, prefix, extendee_spec, monkeypatch): - ext_spec = spack.spec.Spec(name) - ext_spec._concrete = True - - ext_spec.package.spec.prefix = prefix - ext_pkg = ext_spec.package - - # temporarily override extendee_spec property on the package - monkeypatch.setattr(ext_pkg.__class__, "extendee_spec", extendee_spec) - - return ext_pkg - - -def create_python_ext_pkg(name, prefix, python_spec, monkeypatch, namespace=None): - ext_pkg = create_ext_pkg(name, prefix, python_spec, monkeypatch) - ext_pkg.py_namespace = namespace - return ext_pkg - - -def create_dir_structure(tmpdir, dir_structure): - for fname, children in dir_structure.items(): - tmpdir.ensure(fname, dir=fname.endswith("/")) - if children: - create_dir_structure(tmpdir.join(fname), children) - - -@pytest.fixture() -def builtin_and_mock_packages(): - # These tests use mock_repo packages to test functionality of builtin - # packages for python and perl. To test this we put the mock repo at lower - # precedence than the builtin repo, so we test builtin.perl against - # builtin.mock.perl-extension. - repo_dirs = [spack.paths.packages_path, spack.paths.mock_packages_path] - with spack.repo.use_repositories(*repo_dirs): - yield - - -@pytest.fixture() -def python_and_extension_dirs(tmpdir, builtin_and_mock_packages): - python_dirs = {"bin/": {"python": None}, "lib/": {"python2.7/": {"site-packages/": None}}} - - python_name = "python" - python_prefix = tmpdir.join(python_name) - create_dir_structure(python_prefix, python_dirs) - - python_spec = spack.spec.Spec("python@2.7.12") - python_spec._concrete = True - python_spec.package.spec.prefix = str(python_prefix) - - ext_dirs = { - "bin/": {"py-ext-tool": None}, - "lib/": {"python2.7/": {"site-packages/": {"py-extension1/": {"sample.py": None}}}}, - } - - ext_name = "py-extension1" - ext_prefix = tmpdir.join(ext_name) - create_dir_structure(ext_prefix, ext_dirs) - - easy_install_location = "lib/python2.7/site-packages/easy-install.pth" - with open(str(ext_prefix.join(easy_install_location)), "w") as f: - f.write( - """path/to/ext1.egg -path/to/setuptools.egg""" - ) - - return str(python_prefix), str(ext_prefix) - - -@pytest.fixture() -def namespace_extensions(tmpdir, builtin_and_mock_packages): - ext1_dirs = { - "bin/": {"py-ext-tool1": None}, - "lib/": { - "python2.7/": { - "site-packages/": { - "examplenamespace/": {"__init__.py": None, "ext1_sample.py": None} - } - } - }, - } - - ext2_dirs = { - "bin/": {"py-ext-tool2": None}, - "lib/": { - "python2.7/": { - "site-packages/": { - "examplenamespace/": {"__init__.py": None, "ext2_sample.py": None} - } - } - }, - } - - ext1_name = "py-extension1" - ext1_prefix = tmpdir.join(ext1_name) - create_dir_structure(ext1_prefix, ext1_dirs) - - ext2_name = "py-extension2" - ext2_prefix = tmpdir.join(ext2_name) - create_dir_structure(ext2_prefix, ext2_dirs) - - return str(ext1_prefix), str(ext2_prefix), "examplenamespace" - - -def test_python_activation_with_files( - tmpdir, python_and_extension_dirs, monkeypatch, builtin_and_mock_packages -): - python_prefix, ext_prefix = python_and_extension_dirs - - python_spec = spack.spec.Spec("python@2.7.12") - python_spec._concrete = True - python_spec.package.spec.prefix = python_prefix - - ext_pkg = create_python_ext_pkg("py-extension1", ext_prefix, python_spec, monkeypatch) - - python_pkg = python_spec.package - python_pkg.activate(ext_pkg, python_pkg.view()) - - assert os.path.exists(os.path.join(python_prefix, "bin/py-ext-tool")) - - easy_install_location = "lib/python2.7/site-packages/easy-install.pth" - with open(os.path.join(python_prefix, easy_install_location), "r") as f: - easy_install_contents = f.read() - - assert "ext1.egg" in easy_install_contents - assert "setuptools.egg" not in easy_install_contents - - -def test_python_activation_view( - tmpdir, python_and_extension_dirs, builtin_and_mock_packages, monkeypatch -): - python_prefix, ext_prefix = python_and_extension_dirs - - python_spec = spack.spec.Spec("python@2.7.12") - python_spec._concrete = True - python_spec.package.spec.prefix = python_prefix - - ext_pkg = create_python_ext_pkg("py-extension1", ext_prefix, python_spec, monkeypatch) - - view_dir = str(tmpdir.join("view")) - layout = DirectoryLayout(view_dir) - view = YamlFilesystemView(view_dir, layout) - - python_pkg = python_spec.package - python_pkg.activate(ext_pkg, view) - - assert not os.path.exists(os.path.join(python_prefix, "bin/py-ext-tool")) - - assert os.path.exists(os.path.join(view_dir, "bin/py-ext-tool")) - - -def test_python_ignore_namespace_init_conflict( - tmpdir, namespace_extensions, builtin_and_mock_packages, monkeypatch -): - """Test the view update logic in PythonPackage ignores conflicting - instances of __init__ for packages which are in the same namespace. - """ - ext1_prefix, ext2_prefix, py_namespace = namespace_extensions - - python_spec = spack.spec.Spec("python@2.7.12") - python_spec._concrete = True - - ext1_pkg = create_python_ext_pkg( - "py-extension1", ext1_prefix, python_spec, monkeypatch, py_namespace - ) - ext2_pkg = create_python_ext_pkg( - "py-extension2", ext2_prefix, python_spec, monkeypatch, py_namespace - ) - - view_dir = str(tmpdir.join("view")) - layout = DirectoryLayout(view_dir) - view = YamlFilesystemView(view_dir, layout) - - python_pkg = python_spec.package - python_pkg.activate(ext1_pkg, view) - # Normally handled by Package.do_activate, but here we activate directly - view.extensions_layout.add_extension(python_spec, ext1_pkg.spec) - python_pkg.activate(ext2_pkg, view) - - f1 = "lib/python2.7/site-packages/examplenamespace/ext1_sample.py" - f2 = "lib/python2.7/site-packages/examplenamespace/ext2_sample.py" - init_file = "lib/python2.7/site-packages/examplenamespace/__init__.py" - - assert os.path.exists(os.path.join(view_dir, f1)) - assert os.path.exists(os.path.join(view_dir, f2)) - assert os.path.exists(os.path.join(view_dir, init_file)) - - -def test_python_keep_namespace_init( - tmpdir, namespace_extensions, builtin_and_mock_packages, monkeypatch -): - """Test the view update logic in PythonPackage keeps the namespace - __init__ file as long as one package in the namespace still - exists. - """ - ext1_prefix, ext2_prefix, py_namespace = namespace_extensions - - python_spec = spack.spec.Spec("python@2.7.12") - python_spec._concrete = True - - ext1_pkg = create_python_ext_pkg( - "py-extension1", ext1_prefix, python_spec, monkeypatch, py_namespace - ) - ext2_pkg = create_python_ext_pkg( - "py-extension2", ext2_prefix, python_spec, monkeypatch, py_namespace - ) - - view_dir = str(tmpdir.join("view")) - layout = DirectoryLayout(view_dir) - view = YamlFilesystemView(view_dir, layout) - - python_pkg = python_spec.package - python_pkg.activate(ext1_pkg, view) - # Normally handled by Package.do_activate, but here we activate directly - view.extensions_layout.add_extension(python_spec, ext1_pkg.spec) - python_pkg.activate(ext2_pkg, view) - view.extensions_layout.add_extension(python_spec, ext2_pkg.spec) - - f1 = "lib/python2.7/site-packages/examplenamespace/ext1_sample.py" - init_file = "lib/python2.7/site-packages/examplenamespace/__init__.py" - - python_pkg.deactivate(ext1_pkg, view) - view.extensions_layout.remove_extension(python_spec, ext1_pkg.spec) - - assert not os.path.exists(os.path.join(view_dir, f1)) - assert os.path.exists(os.path.join(view_dir, init_file)) - - python_pkg.deactivate(ext2_pkg, view) - view.extensions_layout.remove_extension(python_spec, ext2_pkg.spec) - - assert not os.path.exists(os.path.join(view_dir, init_file)) - - -def test_python_namespace_conflict( - tmpdir, namespace_extensions, monkeypatch, builtin_and_mock_packages -): - """Test the view update logic in PythonPackage reports an error when two - python extensions with different namespaces have a conflicting __init__ - file. - """ - ext1_prefix, ext2_prefix, py_namespace = namespace_extensions - other_namespace = py_namespace + "other" - - python_spec = spack.spec.Spec("python@2.7.12") - python_spec._concrete = True - - ext1_pkg = create_python_ext_pkg( - "py-extension1", ext1_prefix, python_spec, monkeypatch, py_namespace - ) - ext2_pkg = create_python_ext_pkg( - "py-extension2", ext2_prefix, python_spec, monkeypatch, other_namespace - ) - - view_dir = str(tmpdir.join("view")) - layout = DirectoryLayout(view_dir) - view = YamlFilesystemView(view_dir, layout) - - python_pkg = python_spec.package - python_pkg.activate(ext1_pkg, view) - view.extensions_layout.add_extension(python_spec, ext1_pkg.spec) - with pytest.raises(MergeConflictError): - python_pkg.activate(ext2_pkg, view) - - -@pytest.fixture() -def perl_and_extension_dirs(tmpdir, builtin_and_mock_packages): - perl_dirs = { - "bin/": {"perl": None}, - "lib/": {"site_perl/": {"5.24.1/": {"x86_64-linux/": None}}}, - } - - perl_name = "perl" - perl_prefix = tmpdir.join(perl_name) - create_dir_structure(perl_prefix, perl_dirs) - - perl_spec = spack.spec.Spec("perl@5.24.1") - perl_spec._concrete = True - perl_spec.package.spec.prefix = str(perl_prefix) - - ext_dirs = { - "bin/": {"perl-ext-tool": None}, - "lib/": {"site_perl/": {"5.24.1/": {"x86_64-linux/": {"TestExt/": {}}}}}, - } - - ext_name = "perl-extension" - ext_prefix = tmpdir.join(ext_name) - create_dir_structure(ext_prefix, ext_dirs) - - return str(perl_prefix), str(ext_prefix) - - -def test_perl_activation(tmpdir, builtin_and_mock_packages, monkeypatch): - # Note the lib directory is based partly on the perl version - perl_spec = spack.spec.Spec("perl@5.24.1") - perl_spec._concrete = True - - perl_name = "perl" - tmpdir.ensure(perl_name, dir=True) - - perl_prefix = str(tmpdir.join(perl_name)) - # Set the prefix on the package's spec reference because that is a copy of - # the original spec - perl_spec.package.spec.prefix = perl_prefix - - ext_name = "perl-extension" - tmpdir.ensure(ext_name, dir=True) - ext_pkg = create_ext_pkg(ext_name, str(tmpdir.join(ext_name)), perl_spec, monkeypatch) - - perl_pkg = perl_spec.package - perl_pkg.activate(ext_pkg, perl_pkg.view()) - - -def test_perl_activation_with_files( - tmpdir, perl_and_extension_dirs, monkeypatch, builtin_and_mock_packages -): - perl_prefix, ext_prefix = perl_and_extension_dirs - - perl_spec = spack.spec.Spec("perl@5.24.1") - perl_spec._concrete = True - perl_spec.package.spec.prefix = perl_prefix - - ext_pkg = create_ext_pkg("perl-extension", ext_prefix, perl_spec, monkeypatch) - - perl_pkg = perl_spec.package - perl_pkg.activate(ext_pkg, perl_pkg.view()) - - assert os.path.exists(os.path.join(perl_prefix, "bin/perl-ext-tool")) - - -def test_perl_activation_view( - tmpdir, perl_and_extension_dirs, monkeypatch, builtin_and_mock_packages -): - perl_prefix, ext_prefix = perl_and_extension_dirs - - perl_spec = spack.spec.Spec("perl@5.24.1") - perl_spec._concrete = True - perl_spec.package.spec.prefix = perl_prefix - - ext_pkg = create_ext_pkg("perl-extension", ext_prefix, perl_spec, monkeypatch) - - view_dir = str(tmpdir.join("view")) - layout = DirectoryLayout(view_dir) - view = YamlFilesystemView(view_dir, layout) - - perl_pkg = perl_spec.package - perl_pkg.activate(ext_pkg, view) - - assert not os.path.exists(os.path.join(perl_prefix, "bin/perl-ext-tool")) - - assert os.path.exists(os.path.join(view_dir, "bin/perl-ext-tool")) - - -def test_is_activated_upstream_extendee(tmpdir, builtin_and_mock_packages, monkeypatch): - """When an extendee is installed upstream, make sure that the extension - spec is never considered to be globally activated for it. - """ - extendee_spec = spack.spec.Spec("python") - extendee_spec._concrete = True - - python_name = "python" - tmpdir.ensure(python_name, dir=True) - - python_prefix = str(tmpdir.join(python_name)) - # Set the prefix on the package's spec reference because that is a copy of - # the original spec - extendee_spec.package.spec.prefix = python_prefix - monkeypatch.setattr(extendee_spec.__class__, "installed_upstream", True) - - ext_name = "py-extension1" - tmpdir.ensure(ext_name, dir=True) - ext_pkg = create_ext_pkg(ext_name, str(tmpdir.join(ext_name)), extendee_spec, monkeypatch) - - # The view should not be checked at all if the extendee is installed - # upstream, so use 'None' here - mock_view = None - assert not ext_pkg.is_activated(mock_view) diff --git a/lib/spack/spack/test/views.py b/lib/spack/spack/test/views.py index d2801f12ba..6f9283d9f8 100644 --- a/lib/spack/spack/test/views.py +++ b/lib/spack/spack/test/views.py @@ -3,7 +3,6 @@ # # SPDX-License-Identifier: (Apache-2.0 OR MIT) -import os import sys import pytest @@ -14,26 +13,6 @@ from spack.spec import Spec @pytest.mark.skipif(sys.platform == "win32", reason="Not supported on Windows (yet)") -def test_global_activation(install_mockery, mock_fetch): - """This test ensures that views which are maintained inside of an extendee - package's prefix are maintained as expected and are compatible with - global activations prior to #7152. - """ - spec = Spec("extension1").concretized() - pkg = spec.package - pkg.do_install() - pkg.do_activate() - - extendee_spec = spec["extendee"] - extendee_pkg = spec["extendee"].package - view = extendee_pkg.view() - assert pkg.is_activated(view) - - expected_path = os.path.join(extendee_spec.prefix, ".spack", "extensions.yaml") - assert view.extensions_layout.extension_file_path(extendee_spec) == expected_path - - -@pytest.mark.skipif(sys.platform == "win32", reason="Not supported on Windows (yet)") def test_remove_extensions_ordered(install_mockery, mock_fetch, tmpdir): view_dir = str(tmpdir.join("view")) layout = DirectoryLayout(view_dir) diff --git a/lib/spack/spack/verify.py b/lib/spack/spack/verify.py index 9d910bc43d..e18c98373d 100644 --- a/lib/spack/spack/verify.py +++ b/lib/spack/spack/verify.py @@ -162,39 +162,12 @@ def check_spec_manifest(spec): results.add_error(prefix, "manifest corrupted") return results - # Get extensions active in spec - view = spack.filesystem_view.YamlFilesystemView(prefix, spack.store.layout) - active_exts = view.extensions_layout.extension_map(spec).values() - ext_file = "" - if active_exts: - # No point checking contents of this file as it is the only source of - # truth for that information. - ext_file = view.extensions_layout.extension_file_path(spec) - - def is_extension_artifact(p): - if os.path.islink(p): - if any(os.readlink(p).startswith(e.prefix) for e in active_exts): - # This file is linked in by an extension. Belongs to extension - return True - elif os.path.isdir(p) and p not in manifest: - if all(is_extension_artifact(os.path.join(p, f)) for f in os.listdir(p)): - return True - return False - for root, dirs, files in os.walk(prefix): for entry in list(dirs + files): path = os.path.join(root, entry) - # Do not check links from prefix to active extension - # TODO: make this stricter for non-linux systems that use symlink - # permissions - # Do not check directories that only exist for extensions - if is_extension_artifact(path): - continue - # Do not check manifest file. Can't store your own hash - # Nothing to check for ext_file - if path == manifest_file or path == ext_file: + if path == manifest_file: continue data = manifest.pop(path, {}) diff --git a/share/spack/spack-completion.bash b/share/spack/spack-completion.bash index 2461a0b7c7..1ed8037d2b 100755 --- a/share/spack/spack-completion.bash +++ b/share/spack/spack-completion.bash @@ -337,16 +337,7 @@ _spack() { then SPACK_COMPREPLY="-h --help -H --all-help --color -c --config -C --config-scope -d --debug --timestamp --pdb -e --env -D --env-dir -E --no-env --use-env-repo -k --insecure -l --enable-locks -L --disable-locks -m --mock -b --bootstrap -p --profile --sorted-profile --lines -v --verbose --stacktrace --backtrace -V --version --print-shell-vars" else - SPACK_COMPREPLY="activate add arch audit blame bootstrap build-env buildcache cd change checksum ci clean clone commands compiler compilers concretize config containerize create deactivate debug dependencies dependents deprecate dev-build develop diff docs edit env extensions external fetch find gc gpg graph help info install license list load location log-parse maintainers make-installer mark mirror module patch pkg providers pydoc python reindex remove rm repo resource restage solve spec stage style tags test test-env tutorial undevelop uninstall unit-test unload url verify versions view" - fi -} - -_spack_activate() { - if $list_options - then - SPACK_COMPREPLY="-h --help -f --force -v --view" - else - _installed_packages + SPACK_COMPREPLY="add arch audit blame bootstrap build-env buildcache cd change checksum ci clean clone commands compiler compilers concretize config containerize create debug dependencies dependents deprecate dev-build develop diff docs edit env extensions external fetch find gc gpg graph help info install license list load location log-parse maintainers make-installer mark mirror module patch pkg providers pydoc python reindex remove rm repo resource restage solve spec stage style tags test test-env tutorial undevelop uninstall unit-test unload url verify versions view" fi } @@ -838,15 +829,6 @@ _spack_create() { fi } -_spack_deactivate() { - if $list_options - then - SPACK_COMPREPLY="-h --help -f --force -v --view -a --all" - else - _installed_packages - fi -} - _spack_debug() { if $list_options then @@ -1039,7 +1021,7 @@ _spack_env_depfile() { _spack_extensions() { if $list_options then - SPACK_COMPREPLY="-h --help -l --long -L --very-long -d --deps -p --paths -s --show -v --view" + SPACK_COMPREPLY="-h --help -l --long -L --very-long -d --deps -p --paths -s --show" else _extensions fi diff --git a/var/spack/repos/builtin/packages/perl/package.py b/var/spack/repos/builtin/packages/perl/package.py index b268c86f2b..76ee382ef4 100644 --- a/var/spack/repos/builtin/packages/perl/package.py +++ b/var/spack/repos/builtin/packages/perl/package.py @@ -482,28 +482,6 @@ class Perl(Package): # Perl doesn't use Autotools, it should subclass Package return match_predicate(ignore_arg, patterns) - def activate(self, ext_pkg, view, **args): - ignore = self.perl_ignore(ext_pkg, args) - args.update(ignore=ignore) - - super(Perl, self).activate(ext_pkg, view, **args) - - extensions_layout = view.extensions_layout - exts = extensions_layout.extension_map(self.spec) - exts[ext_pkg.name] = ext_pkg.spec - - def deactivate(self, ext_pkg, view, **args): - ignore = self.perl_ignore(ext_pkg, args) - args.update(ignore=ignore) - - super(Perl, self).deactivate(ext_pkg, view, **args) - - extensions_layout = view.extensions_layout - exts = extensions_layout.extension_map(self.spec) - # Make deactivate idempotent - if ext_pkg.name in exts: - del exts[ext_pkg.name] - @property def command(self): """Returns the Perl command, which may vary depending on the version diff --git a/var/spack/repos/builtin/packages/python/package.py b/var/spack/repos/builtin/packages/python/package.py index 1681e2524e..91e635b38a 100644 --- a/var/spack/repos/builtin/packages/python/package.py +++ b/var/spack/repos/builtin/packages/python/package.py @@ -18,7 +18,7 @@ from llnl.util.filesystem import ( is_nonsymlink_exe_with_shebang, path_contains_subdirectory, ) -from llnl.util.lang import dedupe, match_predicate +from llnl.util.lang import dedupe from spack.build_environment import dso_suffix, stat_suffix from spack.package import * @@ -1401,10 +1401,6 @@ config.update(get_paths()) return path.replace(prefix, "") return os.path.join("include", "python{}".format(self.version.up_to(2))) - @property - def easy_install_file(self): - return join_path(self.purelib, "easy-install.pth") - def setup_run_environment(self, env): env.prepend_path("CPATH", os.pathsep.join(self.spec["python"].headers.directories)) @@ -1526,108 +1522,6 @@ config.update(get_paths()) mkdirp(module.python_platlib) mkdirp(module.python_purelib) - # ======================================================================== - # Handle specifics of activating and deactivating python modules. - # ======================================================================== - - def python_ignore(self, ext_pkg, args): - """Add some ignore files to activate/deactivate args.""" - ignore_arg = args.get("ignore", lambda f: False) - - # Always ignore easy-install.pth, as it needs to be merged. - patterns = [r"(site|dist)-packages/easy-install\.pth$"] - - # Ignore pieces of setuptools installed by other packages. - # Must include directory name or it will remove all site*.py files. - if ext_pkg.name != "py-setuptools": - patterns.extend( - [ - r"bin/easy_install[^/]*$", - r"(site|dist)-packages/setuptools[^/]*\.egg$", - r"(site|dist)-packages/setuptools\.pth$", - r"(site|dist)-packages/site[^/]*\.pyc?$", - r"(site|dist)-packages/__pycache__/site[^/]*\.pyc?$", - ] - ) - if ext_pkg.name != "py-pygments": - patterns.append(r"bin/pygmentize$") - if ext_pkg.name != "py-numpy": - patterns.append(r"bin/f2py[0-9.]*$") - - return match_predicate(ignore_arg, patterns) - - def write_easy_install_pth(self, exts, prefix=None): - if not prefix: - prefix = self.prefix - - paths = [] - unique_paths = set() - - for ext in sorted(exts.values()): - easy_pth = join_path(ext.prefix, self.easy_install_file) - - if not os.path.isfile(easy_pth): - continue - - with open(easy_pth) as f: - for line in f: - line = line.rstrip() - - # Skip lines matching these criteria - if not line: - continue - if re.search(r"^(import|#)", line): - continue - if ext.name != "py-setuptools" and re.search(r"setuptools.*egg$", line): - continue - - if line not in unique_paths: - unique_paths.add(line) - paths.append(line) - - main_pth = join_path(prefix, self.easy_install_file) - - if not paths: - if os.path.isfile(main_pth): - os.remove(main_pth) - - else: - with open(main_pth, "w") as f: - f.write("import sys; sys.__plen = len(sys.path)\n") - for path in paths: - f.write("{0}\n".format(path)) - f.write( - "import sys; new=sys.path[sys.__plen:]; " - "del sys.path[sys.__plen:]; " - "p=getattr(sys,'__egginsert',0); " - "sys.path[p:p]=new; " - "sys.__egginsert = p+len(new)\n" - ) - - def activate(self, ext_pkg, view, **args): - ignore = self.python_ignore(ext_pkg, args) - args.update(ignore=ignore) - - super(Python, self).activate(ext_pkg, view, **args) - - extensions_layout = view.extensions_layout - exts = extensions_layout.extension_map(self.spec) - exts[ext_pkg.name] = ext_pkg.spec - - self.write_easy_install_pth(exts, prefix=view.get_projection_for_spec(self.spec)) - - def deactivate(self, ext_pkg, view, **args): - args.update(ignore=self.python_ignore(ext_pkg, args)) - - super(Python, self).deactivate(ext_pkg, view, **args) - - extensions_layout = view.extensions_layout - exts = extensions_layout.extension_map(self.spec) - # Make deactivate idempotent - if ext_pkg.name in exts: - del exts[ext_pkg.name] - self.write_easy_install_pth(exts, prefix=view.get_projection_for_spec(self.spec)) - def add_files_to_view(self, view, merge_map, skip_if_exists=True): bin_dir = self.spec.prefix.bin if sys.platform != "win32" else self.spec.prefix for src, dst in merge_map.items(): |