diff options
author | Massimiliano Culpo <massimiliano.culpo@gmail.com> | 2019-10-03 07:15:01 +0200 |
---|---|---|
committer | Todd Gamblin <tgamblin@llnl.gov> | 2019-10-02 22:15:01 -0700 |
commit | 76b9c561109e4a19375231fd6e8fd8a1c7947f34 (patch) | |
tree | e68018d5397a30eee2eaa1926a2dcabbc75b5c18 | |
parent | b11984767b79f05d261ef112a46e09bc1939204f (diff) | |
download | spack-76b9c561109e4a19375231fd6e8fd8a1c7947f34.tar.gz spack-76b9c561109e4a19375231fd6e8fd8a1c7947f34.tar.bz2 spack-76b9c561109e4a19375231fd6e8fd8a1c7947f34.tar.xz spack-76b9c561109e4a19375231fd6e8fd8a1c7947f34.zip |
Remove support for generating dotkit files (#11986)
Dotkit is being used only at a few sites and has been deprecated on new
machines. This commit removes all the code that provide support for the
generation of dotkit module files.
A new validator named "deprecatedProperties" has been added to the
jsonschema validators. It permits to prompt a warning message or exit
with an error if a property that has been marked as deprecated is
encountered.
* Removed references to dotkit in the docs
* Removed references to dotkit in setup-env-test.sh
* Added a unit test for the 'deprecatedProperties' schema validator
34 files changed, 121 insertions, 426 deletions
diff --git a/etc/spack/defaults/config.yaml b/etc/spack/defaults/config.yaml index 1b6d83f359..d6c9551d93 100644 --- a/etc/spack/defaults/config.yaml +++ b/etc/spack/defaults/config.yaml @@ -32,7 +32,6 @@ config: module_roots: tcl: $spack/share/spack/modules lmod: $spack/share/spack/lmod - dotkit: $spack/share/spack/dotkit # Temporary locations Spack can try to use for builds. diff --git a/etc/spack/defaults/modules.yaml b/etc/spack/defaults/modules.yaml index b5ea65da95..1d91b0c88c 100644 --- a/etc/spack/defaults/modules.yaml +++ b/etc/spack/defaults/modules.yaml @@ -16,7 +16,6 @@ modules: enable: - tcl - - dotkit prefix_inspections: bin: - PATH diff --git a/lib/spack/docs/basic_usage.rst b/lib/spack/docs/basic_usage.rst index 1c61a38899..f6ceb2a9e6 100644 --- a/lib/spack/docs/basic_usage.rst +++ b/lib/spack/docs/basic_usage.rst @@ -1190,24 +1190,15 @@ Using Extensions ^^^^^^^^^^^^^^^^ There are three ways to get ``numpy`` working in Python. The first is -to use :ref:`shell-support`. You can simply ``use`` or ``load`` the +to use :ref:`shell-support`. You can simply ``load`` the module for the extension, and it will be added to the ``PYTHONPATH`` -in your current shell. - -For tcl modules: +in your current shell: .. code-block:: console $ spack load python $ spack load py-numpy -or, for dotkit: - -.. code-block:: console - - $ spack use python - $ spack use py-numpy - Now ``import numpy`` will succeed for as long as you keep your current session open. diff --git a/lib/spack/docs/config_yaml.rst b/lib/spack/docs/config_yaml.rst index 92030b8ac9..53398f55c2 100644 --- a/lib/spack/docs/config_yaml.rst +++ b/lib/spack/docs/config_yaml.rst @@ -74,7 +74,6 @@ the location for each type of module. e.g.: module_roots: tcl: $spack/share/spack/modules lmod: $spack/share/spack/lmod - dotkit: $spack/share/spack/dotkit See :ref:`modules` for details. diff --git a/lib/spack/docs/configuration.rst b/lib/spack/docs/configuration.rst index c89a90f2e9..7721c07a8e 100644 --- a/lib/spack/docs/configuration.rst +++ b/lib/spack/docs/configuration.rst @@ -463,7 +463,6 @@ account all scopes. For example, to see the fully merged module_roots: tcl: $spack/share/spack/modules lmod: $spack/share/spack/lmod - dotkit: $spack/share/spack/dotkit build_stage: - $tempdir/$user/spack-stage - ~/.spack/stage @@ -514,7 +513,6 @@ down the problem: /home/myuser/spack/etc/spack/defaults/config.yaml:32 module_roots: /home/myuser/spack/etc/spack/defaults/config.yaml:33 tcl: $spack/share/spack/modules /home/myuser/spack/etc/spack/defaults/config.yaml:34 lmod: $spack/share/spack/lmod - /home/myuser/spack/etc/spack/defaults/config.yaml:35 dotkit: $spack/share/spack/dotkit /home/myuser/spack/etc/spack/defaults/config.yaml:49 build_stage: /home/myuser/spack/etc/spack/defaults/config.yaml:50 - $tempdir/$user/spack-stage /home/myuser/spack/etc/spack/defaults/config.yaml:51 - ~/.spack/stage diff --git a/lib/spack/docs/getting_started.rst b/lib/spack/docs/getting_started.rst index ec3dc51eed..eeeffcc25f 100644 --- a/lib/spack/docs/getting_started.rst +++ b/lib/spack/docs/getting_started.rst @@ -71,7 +71,7 @@ This automatically adds Spack to your ``PATH`` and allows the ``spack`` command to be used to execute spack :ref:`commands <shell-support>` and :ref:`useful packaging commands <packaging-shell-support>`. -If :ref:`environment-modules or dotkit <InstallEnvironmentModules>` is +If :ref:`environment-modules <InstallEnvironmentModules>` is installed and available, the ``spack`` command can also load and unload :ref:`modules <modules>`. diff --git a/lib/spack/docs/module_file_support.rst b/lib/spack/docs/module_file_support.rst index f6dbdf3f19..7ce2398af2 100644 --- a/lib/spack/docs/module_file_support.rst +++ b/lib/spack/docs/module_file_support.rst @@ -13,8 +13,8 @@ The use of module systems to manage user environment in a controlled way is a common practice at HPC centers that is often embraced also by individual programmers on their development machines. To support this common practice Spack integrates with `Environment Modules -<http://modules.sourceforge.net/>`_ , `LMod -<http://lmod.readthedocs.io/en/latest/>`_ and `Dotkit <https://computing.llnl.gov/?set=jobs&page=dotkit>`_ by +<http://modules.sourceforge.net/>`_ and `LMod +<http://lmod.readthedocs.io/en/latest/>`_ by providing post-install hooks that generate module files and commands to manipulate them. .. note:: @@ -67,7 +67,7 @@ to load the ``cmake`` module: $ module load cmake-3.7.2-gcc-6.3.0-fowuuby Neither of these is particularly pretty, easy to remember, or -easy to type. Luckily, Spack has its own interface for using modules and dotkits. +easy to type. Luckily, Spack has its own interface for using modules. ^^^^^^^^^^^^^ Shell support @@ -108,20 +108,10 @@ that the startup time may be slightly increased because of that. ^^^^^^^^^^^^^^^^^^^^^^^ Once you have shell support enabled you can use the same spec syntax -you're used to: +you're used to and you can use the same shortened names you use +everywhere else in Spack. -========================= ========================== -Modules Dotkit -========================= ========================== -``spack load <spec>`` ``spack use <spec>`` -``spack unload <spec>`` ``spack unuse <spec>`` -========================= ========================== - -And you can use the same shortened names you use everywhere else in -Spack. - -For example, if you are using dotkit, this will add the ``mpich`` -package built with ``gcc`` to your path: +For example this will add the ``mpich`` package built with ``gcc`` to your path: .. code-block:: console @@ -129,17 +119,10 @@ package built with ``gcc`` to your path: # ... wait for install ... - $ spack use mpich %gcc@4.4.7 # dotkit - Prepending: mpich@3.0.4%gcc@4.4.7 (ok) + $ spack load mpich %gcc@4.4.7 # modules $ which mpicc ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/mpich@3.0.4/bin/mpicc -Or, similarly if you are using modules, you could type: - -.. code-block:: console - - $ spack load mpich %gcc@4.4.7 # modules - These commands will add appropriate directories to your ``PATH``, ``MANPATH``, ``CPATH``, and ``LD_LIBRARY_PATH``. When you no longer want to use a package, you can type unload or unuse similarly: @@ -147,13 +130,12 @@ want to use a package, you can type unload or unuse similarly: .. code-block:: console $ spack unload mpich %gcc@4.4.7 # modules - $ spack unuse mpich %gcc@4.4.7 # dotkit .. note:: - These ``use``, ``unuse``, ``load``, and ``unload`` subcommands are + The ``load`` and ``unload`` subcommands are only available if you have enabled Spack's shell support *and* you - have dotkit or modules installed on your machine. + have environment-modules installed on your machine. ^^^^^^^^^^^^^^^^^^^^^^ Ambiguous module names @@ -292,8 +274,6 @@ that can be generated by Spack: +-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+ | | **Hook name** | **Default root directory** | **Default template file** | **Compatible tools** | +=============================+====================+===============================+==============================================+======================+ - | **Dotkit** | ``dotkit`` | share/spack/dotkit | share/spack/templates/modules/modulefile.dk | DotKit | - +-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+ | **TCL - Non-Hierarchical** | ``tcl`` | share/spack/modules | share/spack/templates/modules/modulefile.tcl | Env. Modules/LMod | +-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+ | **Lua - Hierarchical** | ``lmod`` | share/spack/lmod | share/spack/templates/modules/modulefile.lua | LMod | @@ -377,7 +357,7 @@ are named ``modules.yaml``. The default configuration: .. literalinclude:: _spack_root/etc/spack/defaults/modules.yaml :language: yaml -activates the hooks to generate ``tcl`` and ``dotkit`` module files and inspects +activates the hooks to generate ``tcl`` module files and inspects the installation folder of each package for the presence of a set of subdirectories (``bin``, ``man``, ``share/man``, etc.). If any is found its full path is prepended to the environment variables listed below the folder name. @@ -399,12 +379,9 @@ to the generator being customized: modules: enable: - tcl - - dotkit - lmod tcl: # contains environment modules specific customizations - dotkit: - # contains dotkit specific customizations lmod: # contains lmod specific customizations @@ -590,15 +567,14 @@ do so by using the environment blacklist: .. code-block:: yaml modules: - dotkit: + tcl: all: filter: # Exclude changes to any of these variables environment_blacklist: ['CPATH', 'LIBRARY_PATH'] -The configuration above will generate dotkit module files that will not contain -modifications to either ``CPATH`` or ``LIBRARY_PATH`` and environment module -files that instead will contain these modifications. +The configuration above will generate module files that will not contain +modifications to either ``CPATH`` or ``LIBRARY_PATH``. .. _autoloading-dependencies: diff --git a/lib/spack/spack/cmd/module.py b/lib/spack/spack/cmd/module.py index b337e454ef..86a27786da 100644 --- a/lib/spack/spack/cmd/module.py +++ b/lib/spack/spack/cmd/module.py @@ -7,7 +7,6 @@ import argparse import llnl.util.tty as tty -import spack.cmd.modules.dotkit import spack.cmd.modules.lmod import spack.cmd.modules.tcl @@ -23,7 +22,6 @@ _deprecated_commands = ('refresh', 'find', 'rm', 'loads') def setup_parser(subparser): sp = subparser.add_subparsers(metavar='SUBCOMMAND', dest='module_command') - spack.cmd.modules.dotkit.add_command(sp, _subcommands) spack.cmd.modules.lmod.add_command(sp, _subcommands) spack.cmd.modules.tcl.add_command(sp, _subcommands) diff --git a/lib/spack/spack/cmd/modules/__init__.py b/lib/spack/spack/cmd/modules/__init__.py index 33367f6c88..4f1e640f6c 100644 --- a/lib/spack/spack/cmd/modules/__init__.py +++ b/lib/spack/spack/cmd/modules/__init__.py @@ -137,7 +137,6 @@ def loads(module_type, specs, args, out=sys.stdout): module_commands = { 'tcl': 'module load ', 'lmod': 'module load ', - 'dotkit': 'use ' } d = { diff --git a/lib/spack/spack/cmd/modules/dotkit.py b/lib/spack/spack/cmd/modules/dotkit.py deleted file mode 100644 index fae242a199..0000000000 --- a/lib/spack/spack/cmd/modules/dotkit.py +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -import functools - -import spack.cmd.modules - - -def add_command(parser, command_dict): - dotkit_parser = parser.add_parser( - 'dotkit', help='manipulate dotkit module files' - ) - spack.cmd.modules.setup_parser(dotkit_parser) - - command_dict['dotkit'] = functools.partial( - spack.cmd.modules.modules_cmd, module_type='dotkit' - ) diff --git a/lib/spack/spack/cmd/unuse.py b/lib/spack/spack/cmd/unuse.py deleted file mode 100644 index 930fa316d0..0000000000 --- a/lib/spack/spack/cmd/unuse.py +++ /dev/null @@ -1,23 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -import argparse -from spack.cmd.common import print_module_placeholder_help - -description = "remove package from environment using dotkit" -section = "modules" -level = "long" - - -def setup_parser(subparser): - """Parser is only constructed so that this prints a nice help - message with -h. """ - subparser.add_argument( - 'spec', nargs=argparse.REMAINDER, - help='spec of package to unuse with dotkit') - - -def unuse(parser, args): - print_module_placeholder_help() diff --git a/lib/spack/spack/cmd/use.py b/lib/spack/spack/cmd/use.py deleted file mode 100644 index 1a17eb7cf2..0000000000 --- a/lib/spack/spack/cmd/use.py +++ /dev/null @@ -1,24 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -import argparse -from spack.cmd.common import print_module_placeholder_help, arguments - -description = "add package to environment using dotkit" -section = "modules" -level = "long" - - -def setup_parser(subparser): - """Parser is only constructed so that this prints a nice help - message with -h. """ - subparser.add_argument( - 'spec', nargs=argparse.REMAINDER, - help='spec of package to use with dotkit') - arguments.add_common_arguments(subparser, ['recurse_dependencies']) - - -def use(parser, args): - print_module_placeholder_help() diff --git a/lib/spack/spack/hooks/__init__.py b/lib/spack/spack/hooks/__init__.py index d17e25b1ac..3938f9e6d3 100644 --- a/lib/spack/spack/hooks/__init__.py +++ b/lib/spack/spack/hooks/__init__.py @@ -19,7 +19,7 @@ * post_uninstall(spec) This can be used to implement support for things like module - systems (e.g. modules, dotkit, etc.) or to add other custom + systems (e.g. modules, lmod, etc.) or to add other custom features. """ import os.path diff --git a/lib/spack/spack/main.py b/lib/spack/spack/main.py index c3746b6b26..0984aa3b00 100644 --- a/lib/spack/spack/main.py +++ b/lib/spack/spack/main.py @@ -591,7 +591,6 @@ def print_setup_info(*info): module_roots = spack.config.get('config:module_roots') module_to_roots = { 'tcl': list(), - 'dotkit': list(), 'lmod': list() } for name, path in module_roots.items(): diff --git a/lib/spack/spack/modules/__init__.py b/lib/spack/spack/modules/__init__.py index f1897ab943..20864b5f27 100644 --- a/lib/spack/spack/modules/__init__.py +++ b/lib/spack/spack/modules/__init__.py @@ -4,24 +4,20 @@ # SPDX-License-Identifier: (Apache-2.0 OR MIT) """This package contains code for creating environment modules, which can -include dotkits, TCL non-hierarchical modules, LUA hierarchical modules, and -others. +include TCL non-hierarchical modules, LUA hierarchical modules, and others. """ from __future__ import absolute_import -from .dotkit import DotkitModulefileWriter from .tcl import TclModulefileWriter from .lmod import LmodModulefileWriter __all__ = [ - 'DotkitModulefileWriter', 'TclModulefileWriter', 'LmodModulefileWriter' ] module_types = { - 'dotkit': DotkitModulefileWriter, 'tcl': TclModulefileWriter, 'lmod': LmodModulefileWriter } diff --git a/lib/spack/spack/modules/dotkit.py b/lib/spack/spack/modules/dotkit.py deleted file mode 100644 index 3d8d97770f..0000000000 --- a/lib/spack/spack/modules/dotkit.py +++ /dev/null @@ -1,60 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -"""This module implements the classes necessary to generate dotkit modules.""" -import os.path - -import spack.config -from .common import BaseConfiguration, BaseFileLayout -from .common import BaseContext, BaseModuleFileWriter - -#: Dotkit specific part of the configuration -configuration = spack.config.get('modules:dotkit', {}) - -#: Caches the configuration {spec_hash: configuration} -configuration_registry = {} - - -def make_configuration(spec): - """Returns the dotkit configuration for spec""" - key = spec.dag_hash() - try: - return configuration_registry[key] - except KeyError: - return configuration_registry.setdefault( - key, DotkitConfiguration(spec) - ) - - -def make_layout(spec): - """Returns the layout information for spec """ - conf = make_configuration(spec) - return DotkitFileLayout(conf) - - -def make_context(spec): - """Returns the context information for spec""" - conf = make_configuration(spec) - return DotkitContext(conf) - - -class DotkitConfiguration(BaseConfiguration): - """Configuration class for dotkit module files.""" - - -class DotkitFileLayout(BaseFileLayout): - """File layout for dotkit module files.""" - - #: file extension of dotkit module files - extension = 'dk' - - -class DotkitContext(BaseContext): - """Context class for dotkit module files.""" - - -class DotkitModulefileWriter(BaseModuleFileWriter): - """Writer class for dotkit module files.""" - default_template = os.path.join('modules', 'modulefile.dk') diff --git a/lib/spack/spack/schema/__init__.py b/lib/spack/spack/schema/__init__.py index 3e26238c93..894d1cd743 100644 --- a/lib/spack/spack/schema/__init__.py +++ b/lib/spack/spack/schema/__init__.py @@ -11,6 +11,7 @@ import re import six import llnl.util.lang +import llnl.util.tty import spack.spec @@ -72,11 +73,35 @@ def _make_validator(): '"{0}" is an invalid spec [{1}]'.format(spec_str, str(e)) ) + def _deprecated_properties(validator, deprecated, instance, schema): + if not (validator.is_type(instance, "object") or + validator.is_type(instance, "array")): + return + + # Get a list of the deprecated properties, return if there is none + deprecated_properties = [ + x for x in instance if x in deprecated['properties'] + ] + if not deprecated_properties: + return + + # Retrieve the template message + msg = deprecated['message'] + is_error = deprecated['error'] + if not is_error: + for entry in deprecated_properties: + llnl.util.tty.warn(msg.format(property=entry)) + else: + import jsonschema + for entry in deprecated_properties: + yield jsonschema.ValidationError(msg.format(property=entry)) + return jsonschema.validators.extend( jsonschema.Draft4Validator, { "validate_spec": _validate_spec, "properties": _set_defaults, - "patternProperties": _set_pp_defaults + "patternProperties": _set_pp_defaults, + "deprecatedProperties": _deprecated_properties } ) diff --git a/lib/spack/spack/schema/config.py b/lib/spack/spack/schema/config.py index 31e7f9b283..4c49cbca6a 100644 --- a/lib/spack/spack/schema/config.py +++ b/lib/spack/spack/schema/config.py @@ -41,6 +41,13 @@ properties = { 'lmod': {'type': 'string'}, 'dotkit': {'type': 'string'}, }, + 'deprecatedProperties': { + 'properties': ['dotkit'], + 'message': 'specifying a "{property}" module root has no ' + 'effect [support for {property} module files' + ' has been dropped]', + 'error': False + }, }, 'source_cache': {'type': 'string'}, 'misc_cache': {'type': 'string'}, diff --git a/lib/spack/spack/schema/modules.py b/lib/spack/spack/schema/modules.py index 1da9dbd586..14f185cfcc 100644 --- a/lib/spack/spack/schema/modules.py +++ b/lib/spack/spack/schema/modules.py @@ -135,7 +135,16 @@ properties = { 'default': [], 'items': { 'type': 'string', - 'enum': ['tcl', 'dotkit', 'lmod']}}, + 'enum': ['tcl', 'dotkit', 'lmod'] + }, + 'deprecatedProperties': { + 'properties': ['dotkit'], + 'message': 'cannot enable "{property}" in modules.yaml ' + '[support for {property} module files has been' + ' dropped]', + 'error': False + }, + }, 'lmod': { 'allOf': [ # Base configuration @@ -164,6 +173,13 @@ properties = { ] }, }, + 'deprecatedProperties': { + 'properties': ['dotkit'], + 'message': 'the section "{property}" in modules.yaml has no effect' + ' [support for {property} module files has been ' + 'dropped]', + 'error': False + }, }, } diff --git a/lib/spack/spack/schema/upstreams.py b/lib/spack/spack/schema/upstreams.py index 4bbbac17be..5925782af2 100644 --- a/lib/spack/spack/schema/upstreams.py +++ b/lib/spack/spack/schema/upstreams.py @@ -20,8 +20,7 @@ properties = { 'type': 'object', 'properties': { 'tcl': {'type': 'string'}, - 'lmod': {'type': 'string'}, - 'dotkit': {'type': 'string'} + 'lmod': {'type': 'string'} } } } diff --git a/lib/spack/spack/test/cmd/module.py b/lib/spack/spack/test/cmd/module.py index 6222401d66..ac8b48a9e8 100644 --- a/lib/spack/spack/test/cmd/module.py +++ b/lib/spack/spack/test/cmd/module.py @@ -21,7 +21,6 @@ def _module_files(module_type, *specs): @pytest.fixture(scope='module', autouse=True) def ensure_module_files_are_there(database): - module('dotkit', 'refresh', '-y') module('tcl', 'refresh', '-y') @@ -39,7 +38,7 @@ def failure_args(request): @pytest.fixture( - params=['dotkit', 'tcl', 'lmod'] + params=['tcl', 'lmod'] ) def module_type(request): return request.param diff --git a/lib/spack/spack/test/data/config/modules.yaml b/lib/spack/spack/test/data/config/modules.yaml index 25fe2088e7..adc16e896b 100644 --- a/lib/spack/spack/test/data/config/modules.yaml +++ b/lib/spack/spack/test/data/config/modules.yaml @@ -16,7 +16,6 @@ modules: enable: - tcl - - dotkit prefix_inspections: bin: - PATH diff --git a/lib/spack/spack/test/data/modules/dotkit/autoload_direct.yaml b/lib/spack/spack/test/data/modules/dotkit/autoload_direct.yaml deleted file mode 100644 index 2636d6265b..0000000000 --- a/lib/spack/spack/test/data/modules/dotkit/autoload_direct.yaml +++ /dev/null @@ -1,5 +0,0 @@ -enable: - - dotkit -dotkit: - all: - autoload: 'direct' diff --git a/lib/spack/spack/test/data/modules/dotkit/override_template.yaml b/lib/spack/spack/test/data/modules/dotkit/override_template.yaml deleted file mode 100644 index 8da581689f..0000000000 --- a/lib/spack/spack/test/data/modules/dotkit/override_template.yaml +++ /dev/null @@ -1,5 +0,0 @@ -enable: - - dotkit -dotkit: - all: - template: 'override_from_modules.txt' diff --git a/lib/spack/spack/test/modules/dotkit.py b/lib/spack/spack/test/modules/dotkit.py deleted file mode 100644 index 951fb38343..0000000000 --- a/lib/spack/spack/test/modules/dotkit.py +++ /dev/null @@ -1,51 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - - -import pytest -import spack.modules.dotkit - -#: Class of the writer tested in this module -writer_cls = spack.modules.dotkit.DotkitModulefileWriter - - -@pytest.mark.usefixtures('config', 'mock_packages') -class TestDotkit(object): - - def test_dotkit(self, modulefile_content, module_configuration): - """Tests the generation of a dotkit file that loads dependencies - automatically. - """ - - module_configuration('autoload_direct') - content = modulefile_content('mpileaks arch=x86-linux') - - assert '#c spack' in content - assert '#d mpileaks @2.3' in content - assert len([x for x in content if 'dk_op' in x]) == 2 - - def test_override_template_in_package( - self, modulefile_content, module_configuration - ): - """Tests overriding a template from and attribute in the package.""" - - module_configuration('autoload_direct') - content = modulefile_content('override-module-templates') - - assert 'Override successful!' in content - - def test_override_template_in_modules_yaml( - self, modulefile_content, module_configuration - ): - """Tests overriding a template from `modules.yaml`""" - - module_configuration('override_template') - - # Check that this takes precedence over an attribute in the package - content = modulefile_content('override-module-templates') - assert 'Override even better!' in content - - content = modulefile_content('mpileaks arch=x86-linux') - assert 'Override even better!' in content diff --git a/lib/spack/spack/test/schema.py b/lib/spack/spack/test/schema.py index 88c824513a..9149b77b37 100644 --- a/lib/spack/spack/test/schema.py +++ b/lib/spack/spack/test/schema.py @@ -114,3 +114,28 @@ def test_schema_validation(meta_schema, config_name): # If this validation throws the test won't pass jsonschema.validate(schema, meta_schema) + + +def test_deprecated_properties(module_suffixes_schema): + # Test that an error is reported when 'error: True' + module_suffixes_schema['deprecatedProperties'] = { + 'properties': ['tcl'], + 'message': '{property} not allowed', + 'error': True + } + v = spack.schema.Validator(module_suffixes_schema) + data = {'tcl': {'all': {'suffixes': {'^python': 'py'}}}} + + with pytest.raises(jsonschema.ValidationError, match='tcl not allowed'): + v.validate(data) + + # Test that just a warning is reported when 'error: False' + module_suffixes_schema['deprecatedProperties'] = { + 'properties': ['tcl'], + 'message': '{property} not allowed', + 'error': False + } + v = spack.schema.Validator(module_suffixes_schema) + data = {'tcl': {'all': {'suffixes': {'^python': 'py'}}}} + # The next validation doesn't raise anymore + v.validate(data) diff --git a/share/spack/csh/spack.csh b/share/spack/csh/spack.csh index 01be616572..ac893d72b1 100644 --- a/share/spack/csh/spack.csh +++ b/share/spack/csh/spack.csh @@ -5,27 +5,27 @@ ######################################################################## # This is a wrapper around the spack command that forwards calls to -# 'spack use' and 'spack unuse' to shell functions. This in turn -# allows them to be used to invoke dotkit functions. +# 'spack load' and 'spack unload' to shell functions. This in turn +# allows them to be used to invoke environment-modules functions. # -# 'spack use' is smarter than just 'use' because it converts its -# arguments into a unique spack spec that is then passed to dotkit +# 'spack load' is smarter than just 'load' because it converts its +# arguments into a unique Spack spec that is then passed to environment-modules # commands. This allows the user to use packages without knowing all # their installation details. # -# e.g., rather than requring a full spec for libelf, the user can type: +# e.g., rather than requiring a full spec for libelf, the user can type: # -# spack use libelf +# spack load libelf # -# This will first find the available libelf dotkits and use a +# This will first find the available libelf module file and use a # matching one. If there are two versions of libelf, the user would # need to be more specific, e.g.: # -# spack use libelf@0.8.13 +# spack load libelf@0.8.13 # # This is very similar to how regular spack commands work and it # avoids the need to come up with a user-friendly naming scheme for -# spack dotfiles. +# spack module files. ######################################################################## # accumulate initial flags for main spack command set _sp_flags = "" @@ -104,8 +104,6 @@ case env: breaksw endsw endif -case use: -case unuse: case load: case unload: set _sp_module_args="""" @@ -115,22 +113,10 @@ case unload: set _sp_spec = ($_sp_spec) endif - # Here the user has run use or unuse with a spec. Find a matching + # Here the user has run load or unload with a spec. Find a matching # spec using 'spack module find', then use the appropriate module # tool's commands to add/remove the result from the environment. switch ($_sp_subcommand) - case "use": - set _sp_full_spec = ( "`\spack $_sp_flags module dotkit find $_sp_spec`" ) - if ( $? == 0 ) then - use $_sp_module_args $_sp_full_spec - endif - breaksw - case "unuse": - set _sp_full_spec = ( "`\spack $_sp_flags module dotkit find $_sp_spec`" ) - if ( $? == 0 ) then - unuse $_sp_module_args $_sp_full_spec - endif - breaksw case "load": # _sp_module_args may be "-r" for recursive spec retrieval set _sp_full_spec = ( "`\spack $_sp_flags module tcl find $_sp_module_args $_sp_spec`" ) diff --git a/share/spack/qa/setup-env-test.sh b/share/spack/qa/setup-env-test.sh index 6515a99d8c..e16e31fffb 100755 --- a/share/spack/qa/setup-env-test.sh +++ b/share/spack/qa/setup-env-test.sh @@ -167,7 +167,7 @@ is_not_set() { # ----------------------------------------------------------------------- -# Instead of invoking the module/use/dotkit commands, we print the +# Instead of invoking the module commands, we print the # arguments that Spack invokes the command with, so we can check that # Spack passes the expected arguments in the tests below. # @@ -177,14 +177,6 @@ module() { echo module "$@" } -use() { - echo use "$@" -} - -unuse() { - echo unuse "$@" -} - # ----------------------------------------------------------------------- # Setup test environment and do some preliminary checks # ----------------------------------------------------------------------- @@ -219,11 +211,9 @@ echo "Creating a mock package installation" spack -m install --fake a a_install=$(spack location -i a) a_module=$(spack -m module tcl find a) -a_dotkit=$(spack -m module dotkit find a) b_install=$(spack location -i b) b_module=$(spack -m module tcl find b) -b_dotkit=$(spack -m module dotkit find b) # create a test environment for tesitng environment commands echo "Creating a mock environment" @@ -304,26 +294,6 @@ contains "usage: spack unload " spack -m unload -h contains "usage: spack unload " spack -m unload -h d contains "usage: spack unload " spack -m unload --help -title 'Testing `spack use`' -contains "use $b_dotkit" spack -m use b -fails spack -m use -l -contains "use -l --arg $b_dotkit" spack -m use -l --arg b -contains "use $b_dotkit $a_dotkit" spack -m use -r a -contains "use $b_dotkit $a_dotkit" spack -m use --dependencies a -fails spack -m use d -contains "usage: spack use " spack -m use -h -contains "usage: spack use " spack -m use -h d -contains "usage: spack use " spack -m use --help - -title 'Testing `spack unuse`' -contains "unuse $b_dotkit" spack -m unuse b -fails spack -m unuse -l -contains "unuse -l --arg $b_dotkit" spack -m unuse -l --arg b -fails spack -m unuse d -contains "usage: spack unuse " spack -m unuse -h -contains "usage: spack unuse " spack -m unuse -h d -contains "usage: spack unuse " spack -m unuse --help - title 'Testing `spack env`' contains "usage: spack env " spack env -h contains "usage: spack env " spack env --help diff --git a/share/spack/setup-env.csh b/share/spack/setup-env.csh index ac5479196d..b49597918c 100755 --- a/share/spack/setup-env.csh +++ b/share/spack/setup-env.csh @@ -6,7 +6,7 @@ # # This file is part of Spack and sets up the spack environment for -# csh and tcsh. This includes dotkit support, module support, and +# csh and tcsh. This includes environment modules and lmod support, and # it also puts spack in your path. Source it like this: # # setenv SPACK_ROOT /path/to/spack @@ -24,7 +24,7 @@ if ($?SPACK_ROOT) then _spack_pathadd PATH "$SPACK_ROOT/bin" eval `spack --print-shell-vars csh` - # Set up modules and dotkit search paths in the user environment + # Set up module search paths in the user environment set tcl_roots = `echo $_sp_tcl_roots:q | sed 's/:/ /g'` set compatible_sys_types = `echo $_sp_compatible_sys_types:q | sed 's/:/ /g'` foreach tcl_root ($tcl_roots:q) @@ -33,10 +33,6 @@ if ($?SPACK_ROOT) then end end - set dotkit_roots = `echo $_sp_dotkit_roots:q | sed 's/:/ /g'` - foreach dotkit_root ($dotkit_roots) - _spack_pathadd DK_NODE "$dotkit_root/$_sp_sys_type" - end else echo "ERROR: Sourcing spack setup-env.csh requires setting SPACK_ROOT to " echo " the root of your spack installation." diff --git a/share/spack/setup-env.sh b/share/spack/setup-env.sh index 0fd5eb30f7..d5cad059a6 100755 --- a/share/spack/setup-env.sh +++ b/share/spack/setup-env.sh @@ -7,8 +7,8 @@ ######################################################################## # # This file is part of Spack and sets up the spack environment for bash, -# zsh, and dash (sh). This includes dotkit support, module support, and -# it also puts spack in your path. The script also checks that at least +# zsh, and dash (sh). This includes environment modules and lmod support, +# and it also puts spack in your path. The script also checks that at least # module support exists, and provides suggestions if it doesn't. Source # it like this: # @@ -16,27 +16,27 @@ # ######################################################################## # This is a wrapper around the spack command that forwards calls to -# 'spack use' and 'spack unuse' to shell functions. This in turn -# allows them to be used to invoke dotkit functions. +# 'spack load' and 'spack unload' to shell functions. This in turn +# allows them to be used to invoke environment modules functions. # -# 'spack use' is smarter than just 'use' because it converts its -# arguments into a unique spack spec that is then passed to dotkit +# 'spack load' is smarter than just 'load' because it converts its +# arguments into a unique Spack spec that is then passed to module # commands. This allows the user to use packages without knowing all # their installation details. # # e.g., rather than requiring a full spec for libelf, the user can type: # -# spack use libelf +# spack load libelf # -# This will first find the available libelf dotkits and use a +# This will first find the available libelf module file and use a # matching one. If there are two versions of libelf, the user would # need to be more specific, e.g.: # -# spack use libelf@0.8.13 +# spack load libelf@0.8.13 # # This is very similar to how regular spack commands work and it # avoids the need to come up with a user-friendly naming scheme for -# spack dotfiles. +# spack module files. ######################################################################## spack() { @@ -140,7 +140,7 @@ spack() { fi return ;; - "use"|"unuse"|"load"|"unload") + "load"|"unload") # Shift any other args for use off before parsing spec. _sp_subcommand_args="" _sp_module_args="" @@ -161,20 +161,6 @@ spack() { # tool's commands to add/remove the result from the environment. # If spack module command comes back with an error, do nothing. case $_sp_subcommand in - "use") - if _sp_full_spec=$(command spack $_sp_flags module dotkit find $_sp_subcommand_args "$@"); then - use $_sp_module_args $_sp_full_spec - else - $(exit 1) - fi - ;; - "unuse") - if _sp_full_spec=$(command spack $_sp_flags module dotkit find $_sp_subcommand_args "$@"); then - unuse $_sp_module_args $_sp_full_spec - else - $(exit 1) - fi - ;; "load") if _sp_full_spec=$(command spack $_sp_flags module tcl find $_sp_subcommand_args "$@"); then module load $_sp_module_args $_sp_full_spec @@ -356,7 +342,6 @@ _sp_multi_pathadd() { done } _sp_multi_pathadd MODULEPATH "$_sp_tcl_roots" -_sp_multi_pathadd DK_NODE "$_sp_dotkit_roots" # Add programmable tab completion for Bash # diff --git a/share/spack/spack-completion.bash b/share/spack/spack-completion.bash index bd6675dd16..935092aedb 100755 --- a/share/spack/spack-completion.bash +++ b/share/spack/spack-completion.bash @@ -760,7 +760,7 @@ function _spack_module { then compgen -W "-h --help" -- "$cur" else - compgen -W "lmod tcl dotkit" -- "$cur" + compgen -W "lmod tcl" -- "$cur" fi } @@ -812,53 +812,6 @@ function _spack_module_tcl_rm { fi } -function _spack_module_dotkit { - if $list_options - then - compgen -W "-h --help" -- "$cur" - else - compgen -W "refresh find rm loads" -- "$cur" - fi -} - - -function _spack_module_dotkit_find { - if $list_options - then - compgen -W "-h --help --full-path -r --dependencies" -- "$cur" - else - compgen -W "$(_installed_packages)" -- "$cur" - fi -} - -function _spack_module_dotkit_loads { - if $list_options - then - compgen -W "-h --help --input-only -p --prefix -x --exclude - -r --dependencies" -- "$cur" - else - compgen -W "$(_installed_packages)" -- "$cur" - fi - -} - -function _spack_module_dotkit_refresh { - if $list_options - then - compgen -W "-h --help --delete-tree -y --yes-to-all" -- "$cur" - else - compgen -W "$(_installed_packages)" -- "$cur" - fi -} - -function _spack_module_dotkit_rm { - if $list_options - then - compgen -W "-h --help -y --yes-to-all" -- "$cur" - else - compgen -W "$(_installed_packages)" -- "$cur" - fi -} function _spack_module_lmod { if $list_options diff --git a/share/spack/templates/modules/modulefile.dk b/share/spack/templates/modules/modulefile.dk deleted file mode 100644 index c3f6d2dce1..0000000000 --- a/share/spack/templates/modules/modulefile.dk +++ /dev/null @@ -1,31 +0,0 @@ -{% block header %} -{% if category %} -#c {{ category }} -{% endif %} -{% if short_description %} -#d {{ short_description }} -{% endif %} -{% if long_description %} -{{ long_description| textwrap(72)| prepend_to_line('#h ')| join() }} -{% endif %} -{% endblock %} - -{% block autoloads %} -{% for module in autoload %} -dk_op {{ module }} -{% endfor %} -{% endblock %} - -{% block environment %} -{% for command_name, cmd in environment_modifications %} -{% if command_name == 'PrependPath' %} -dk_alter {{ cmd.name }} {{ cmd.value }} -{% endif %} -{% if command_name == 'RemovePath' %} -dk_unalter {{ cmd.name }} {{ cmd.value }} -{% endif %} -{% if command_name == 'SetEnv' %} -dk_setenv {{ cmd.name }} {{ cmd.value }} -{% endif %} -{% endfor %} -{% endblock %} diff --git a/var/spack/repos/builtin.mock/packages/override-module-templates/package.py b/var/spack/repos/builtin.mock/packages/override-module-templates/package.py index 818591a15b..f3cad009aa 100644 --- a/var/spack/repos/builtin.mock/packages/override-module-templates/package.py +++ b/var/spack/repos/builtin.mock/packages/override-module-templates/package.py @@ -12,7 +12,6 @@ class OverrideModuleTemplates(Package): version('1.0', 'foobarbaz') - dotkit_template = 'override.txt' tcl_template = 'override.txt' lmod_template = 'override.txt' diff --git a/var/spack/repos/builtin/packages/go/package.py b/var/spack/repos/builtin/packages/go/package.py index 11f2ff7945..e8a20515f9 100644 --- a/var/spack/repos/builtin/packages/go/package.py +++ b/var/spack/repos/builtin/packages/go/package.py @@ -134,6 +134,6 @@ class Go(Package): # This *MUST* be first, this is where new code is installed spack_env.set('GOPATH', ':'.join(path_components)) - # Allow packages to find this when using module or dotkit + # Allow packages to find this when using module files run_env.prepend_path('GOPATH', ':'.join( [dependent_spec.prefix] + path_components)) |