diff options
author | Massimiliano Culpo <massimiliano.culpo@gmail.com> | 2018-07-20 10:30:36 +0200 |
---|---|---|
committer | Todd Gamblin <tgamblin@llnl.gov> | 2018-07-24 11:27:17 -0700 |
commit | 35cef169740d8eb87a1cc76fcde85d207935d91a (patch) | |
tree | 6925bf9fc43c38acd5d5c14ea57ac72a8a2a8973 /lib | |
parent | 7cbe1aedb7c0ecc32d2cee3e965ec71570ee09d9 (diff) | |
download | spack-35cef169740d8eb87a1cc76fcde85d207935d91a.tar.gz spack-35cef169740d8eb87a1cc76fcde85d207935d91a.tar.bz2 spack-35cef169740d8eb87a1cc76fcde85d207935d91a.tar.xz spack-35cef169740d8eb87a1cc76fcde85d207935d91a.zip |
Grouped all the module commands under `spack module`
As requested in the review all the commands meant to manage module
files have been grouped under the `spack module` command.
Unit tests have been refactored to match the new command structure.
Diffstat (limited to 'lib')
-rw-r--r-- | lib/spack/docs/developer_guide.rst | 2 | ||||
-rw-r--r-- | lib/spack/docs/getting_started.rst | 4 | ||||
-rw-r--r-- | lib/spack/docs/module_file_support.rst | 30 | ||||
-rw-r--r-- | lib/spack/docs/tutorial_modules.rst | 103 | ||||
-rw-r--r-- | lib/spack/docs/workflows.rst | 20 | ||||
-rw-r--r-- | lib/spack/spack/cmd/module.py | 45 | ||||
-rw-r--r-- | lib/spack/spack/cmd/modules/__init__.py (renamed from lib/spack/spack/cmd/common/modules.py) | 13 | ||||
-rw-r--r-- | lib/spack/spack/cmd/modules/dotkit.py (renamed from lib/spack/spack/cmd/dotkit.py) | 22 | ||||
-rw-r--r-- | lib/spack/spack/cmd/modules/lmod.py (renamed from lib/spack/spack/cmd/lmod.py) | 42 | ||||
-rw-r--r-- | lib/spack/spack/cmd/modules/tcl.py (renamed from lib/spack/spack/cmd/tcl.py) | 22 | ||||
-rw-r--r-- | lib/spack/spack/test/cmd/dotkit.py | 101 | ||||
-rw-r--r-- | lib/spack/spack/test/cmd/module.py (renamed from lib/spack/spack/test/cmd/lmod.py) | 117 | ||||
-rw-r--r-- | lib/spack/spack/test/cmd/tcl.py | 135 |
13 files changed, 266 insertions, 390 deletions
diff --git a/lib/spack/docs/developer_guide.rst b/lib/spack/docs/developer_guide.rst index c44806bd61..046de145d0 100644 --- a/lib/spack/docs/developer_guide.rst +++ b/lib/spack/docs/developer_guide.rst @@ -314,7 +314,7 @@ See the `Argparse documentation <https://docs.python.org/2.7/library/argparse.ht for more details on how to add arguments. Some commands have a set of subcommands, like ``spack compiler find`` or -``spack lmod refresh``. You can add subparsers to your parser to handle +``spack module lmod refresh``. You can add subparsers to your parser to handle this. Check out ``spack edit --command compiler`` for an example of this. A lot of commands take the same arguments and flags. These arguments should diff --git a/lib/spack/docs/getting_started.rst b/lib/spack/docs/getting_started.rst index dc70dc8dc9..5c26c30d9f 100644 --- a/lib/spack/docs/getting_started.rst +++ b/lib/spack/docs/getting_started.rst @@ -816,7 +816,7 @@ This problem is related to OpenSSL, and in some cases might be solved by installing a new version of ``git`` and ``openssl``: #. Run ``spack install git`` -#. Add the output of ``spack tcl loads git`` to your ``.bashrc``. +#. Add the output of ``spack module tcl loads git`` to your ``.bashrc``. If this doesn't work, it is also possible to disable checking of SSL certificates by using: @@ -861,7 +861,7 @@ or alternately: .. code-block:: console - $ spack tcl loads curl >>~/.bashrc + $ spack module tcl loads curl >>~/.bashrc or if environment modules don't work: diff --git a/lib/spack/docs/module_file_support.rst b/lib/spack/docs/module_file_support.rst index 67e041e061..41e1245d9a 100644 --- a/lib/spack/docs/module_file_support.rst +++ b/lib/spack/docs/module_file_support.rst @@ -180,9 +180,9 @@ To identify just the one built with the Intel compiler. .. _cmd-spack-module-loads: -^^^^^^^^^^^^^^^^^^^^^^ -``spack tcl loads`` -^^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^ +``spack module tcl loads`` +^^^^^^^^^^^^^^^^^^^^^^^^^^ In some cases, it is desirable to load not just a module, but also all the modules it depends on. This is not required for most modules @@ -195,13 +195,13 @@ Scripts to load modules recursively may be made with the command: .. code-block:: console - $ spack tcl loads --dependencies <spec> + $ spack module tcl loads --dependencies <spec> An equivalent alternative using `process substitution <http://tldp.org/LDP/abs/html/process-sub.html>`_ is: .. code-block :: console - $ source <( spack tcl loads --dependencies <spec> ) + $ source <( spack module tcl loads --dependencies <spec> ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -211,12 +211,12 @@ Module Commands for Shell Scripts Although Spack is flexible, the ``module`` command is much faster. This could become an issue when emitting a series of ``spack load`` commands inside a shell script. By adding the ``--shell`` flag, -``spack tcl find`` may also be used to generate code that can be +``spack module tcl find`` may also be used to generate code that can be cut-and-pasted into a shell script. For example: .. code-block:: console - $ spack tcl loads --dependencies py-numpy git + $ spack module tcl loads --dependencies py-numpy git # bzip2@1.0.6%gcc@4.9.3=linux-x86_64 module load bzip2-1.0.6-gcc-4.9.3-ktnrhkrmbbtlvnagfatrarzjojmkvzsx # ncurses@6.0%gcc@4.9.3=linux-x86_64 @@ -256,9 +256,9 @@ Module Prefixes ^^^^^^^^^^^^^^^ On some systems, modules are automatically prefixed with a certain -string; ``spack tcl loads`` needs to know about that prefix when it +string; ``spack module tcl loads`` needs to know about that prefix when it issues ``module load`` commands. Add the ``--prefix`` option to your -``spack tcl loads`` commands if this is necessary. +``spack module tcl loads`` commands if this is necessary. For example, consider the following on one system: @@ -267,11 +267,11 @@ For example, consider the following on one system: $ module avail linux-SuSE11-x86_64/antlr-2.7.7-gcc-5.3.0-bdpl46y - $ spack tcl loads antlr # WRONG! + $ spack module tcl loads antlr # WRONG! # antlr@2.7.7%gcc@5.3.0~csharp+cxx~java~python arch=linux-SuSE11-x86_64 module load antlr-2.7.7-gcc-5.3.0-bdpl46y - $ spack tcl loads --prefix linux-SuSE11-x86_64/ antlr + $ spack module tcl loads --prefix linux-SuSE11-x86_64/ antlr # antlr@2.7.7%gcc@5.3.0~csharp+cxx~java~python arch=linux-SuSE11-x86_64 module load linux-SuSE11-x86_64/antlr-2.7.7-gcc-5.3.0-bdpl46y @@ -626,9 +626,9 @@ Maintaining Module Files Each type of module file has a command with the same name associated with it. The actions these commands permit are usually associated with the maintenance of a production environment. Here's, for instance, -a sample of the features of the ``spack tcl`` command: +a sample of the features of the ``spack module tcl`` command: -.. command-output:: spack tcl --help +.. command-output:: spack module tcl --help .. _cmd-spack-module-refresh: @@ -639,7 +639,7 @@ Refresh the set of modules The subcommand that regenerates module files to update their content or their layout is ``refresh``: -.. command-output:: spack tcl refresh --help +.. command-output:: spack module tcl refresh --help A set of packages can be selected using anonymous specs for the optional ``constraint`` positional argument. Optionally the entire tree can be deleted @@ -654,7 +654,7 @@ Delete module files If instead what you need is just to delete a few module files, then the right subcommand is ``rm``: -.. command-output:: spack tcl rm --help +.. command-output:: spack module tcl rm --help .. note:: We care about your module files! diff --git a/lib/spack/docs/tutorial_modules.rst b/lib/spack/docs/tutorial_modules.rst index ef817a0223..6175479f35 100644 --- a/lib/spack/docs/tutorial_modules.rst +++ b/lib/spack/docs/tutorial_modules.rst @@ -166,7 +166,7 @@ The fastest way to set-up your environment is to :ref:`use a Docker image <workf .. code-block:: console $ docker pull alalazo/spack:module_tutorial - $ docker run --rm -h module-file-tutorial -it alalazo/spack:module_tutorial + $ docker run --rm -h module-file-tutorial -it spack/module-tutorial:latest root@module-file-tutorial:/# If you arrived at this point you should be ready to start, as all the software needed is @@ -185,7 +185,7 @@ pre-installed in the image: -- linux-ubuntu16.04-x86_64 / gcc@7.2.0 ------------------------- bzip2@1.0.6 mpich@3.2.1 netlib-scalapack@2.0.2 netlib-scalapack@2.0.2 openssl@1.0.2o py-scipy@1.1.0 readline@7.0 - cmake@3.11.4 ncurses@6.1 netlib-scalapack@2.0.2 openblas@0.3.0 pkgconf@1.4.2 py-setuptools@39.2.0 sqlite@3.23.1 + cmake@3.12.0 ncurses@6.1 netlib-scalapack@2.0.2 openblas@0.3.0 pkgconf@1.4.2 py-setuptools@39.2.0 sqlite@3.23.1 gdbm@1.14.1 netlib-lapack@3.8.0 netlib-scalapack@2.0.2 openmpi@1.10.2 py-numpy@1.14.3 python@2.7.15 zlib@1.2.11 Go to :ref:`module_file_tutorial_non_hierarchical` to proceed with the tutorial. @@ -364,17 +364,17 @@ If you arrived to this point you should have an environment that looks similar t autoconf-2.69-gcc-5.4.0-cbvv5rj lua-luaposix-33.4.0-gcc-5.4.0-i7w7ynf perl-5.26.2-gcc-5.4.0-n2k4mza automake-1.16.1-gcc-5.4.0-lykrrr4 m4-1.4.18-gcc-5.4.0-3z33ecf pkgconf-1.4.2-gcc-5.4.0-fovrh7a bzip2-1.0.6-gcc-7.2.0-mwamumj mpc-1.1.0-gcc-5.4.0-56lbd3h pkgconf-1.4.2-gcc-7.2.0-yoxwmgb - cmake-3.11.4-gcc-7.2.0-h24ofjs mpfr-4.0.1-gcc-5.4.0-dy5r7hi py-numpy-1.14.3-gcc-7.2.0-t3loxvu + cmake-3.12.0-gcc-7.2.0-6ovorxs mpfr-4.0.1-gcc-5.4.0-dy5r7hi py-numpy-1.14.3-gcc-7.2.0-t3loxvu gcc-7.2.0-gcc-5.4.0-wwhgyej mpich-3.2.1-gcc-7.2.0-vt5xcat py-scipy-1.1.0-gcc-7.2.0-fdiryak gdbm-1.14.1-gcc-5.4.0-q4fpyuo ncurses-6.1-gcc-5.4.0-3o765ou py-setuptools-39.2.0-gcc-7.2.0-jqhycal gdbm-1.14.1-gcc-7.2.0-zk5lhob ncurses-6.1-gcc-7.2.0-xcgzqdv python-2.7.15-gcc-7.2.0-c7pnzul - git-2.9.4-gcc-5.4.0-mkaoyhz netlib-lapack-3.8.0-gcc-7.2.0-bcikpen readline-7.0-gcc-5.4.0-nxhwrg7 - gmp-6.1.2-gcc-5.4.0-qc4qcfz netlib-scalapack-2.0.2-gcc-7.2.0-d3lertf readline-7.0-gcc-7.2.0-ccruj2i - isl-0.19-gcc-5.4.0-hsl7f52 netlib-scalapack-2.0.2-gcc-7.2.0-jae3ilo sqlite-3.23.1-gcc-7.2.0-5ltus3a - libsigsegv-2.11-gcc-5.4.0-fypapcp netlib-scalapack-2.0.2-gcc-7.2.0-lqfhvfh tcl-8.6.8-gcc-5.4.0-qhwyccy - libtool-2.4.6-gcc-5.4.0-o2pfwjf netlib-scalapack-2.0.2-gcc-7.2.0-uhzmwog zlib-1.2.11-gcc-5.4.0-5nus6kn + git-2.9.4-gcc-5.4.0-mkaoyhz netlib-lapack-3.8.0-gcc-7.2.0-7apabqu readline-7.0-gcc-5.4.0-nxhwrg7 + gmp-6.1.2-gcc-5.4.0-qc4qcfz netlib-scalapack-2.0.2-gcc-7.2.0-3bz5rxx readline-7.0-gcc-7.2.0-ccruj2i + isl-0.19-gcc-5.4.0-hsl7f52 netlib-scalapack-2.0.2-gcc-7.2.0-6i5qsqx sqlite-3.23.1-gcc-7.2.0-5ltus3a + libsigsegv-2.11-gcc-5.4.0-fypapcp netlib-scalapack-2.0.2-gcc-7.2.0-uhzmwog tcl-8.6.8-gcc-5.4.0-qhwyccy + libtool-2.4.6-gcc-5.4.0-o2pfwjf netlib-scalapack-2.0.2-gcc-7.2.0-z52ltyy zlib-1.2.11-gcc-5.4.0-5nus6kn lmod-7.7.29-gcc-5.4.0-wl6mywv openblas-0.3.0-gcc-7.2.0-pdatzbi zlib-1.2.11-gcc-7.2.0-ezuwp4p - lua-5.3.4-gcc-5.4.0-izvaota openmpi-1.10.2-gcc-7.2.0-2h6xmxh + lua-5.3.4-gcc-5.4.0-izvaota openmpi-1.10.2-gcc-7.2.0-6oewzwj lua-luafilesystem-1_6_3-gcc-5.4.0-ywlmaou openssl-1.0.2o-gcc-7.2.0-cvldq3v Use "module spider" to find all possible modules. @@ -433,7 +433,7 @@ Next you should regenerate all the module files: .. code-block:: console - root@module-file-tutorial:/# spack tcl refresh + root@module-file-tutorial:/# spack module tcl refresh ==> You are about to regenerate tcl module files for: -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- @@ -444,11 +444,11 @@ Next you should regenerate all the module files: mkaoyhz git@2.9.4 wl6mywv lmod@7.7.29 56lbd3h mpc@1.1.0 nxhwrg7 readline@7.0 -- linux-ubuntu16.04-x86_64 / gcc@7.2.0 ------------------------- - mwamumj bzip2@1.0.6 bcikpen netlib-lapack@3.8.0 pdatzbi openblas@0.3.0 fdiryak py-scipy@1.1.0 ezuwp4p zlib@1.2.11 - h24ofjs cmake@3.11.4 d3lertf netlib-scalapack@2.0.2 2h6xmxh openmpi@1.10.2 jqhycal py-setuptools@39.2.0 + mwamumj bzip2@1.0.6 7apabqu netlib-lapack@3.8.0 pdatzbi openblas@0.3.0 fdiryak py-scipy@1.1.0 ezuwp4p zlib@1.2.11 + 6ovorxs cmake@3.12.0 6i5qsqx netlib-scalapack@2.0.2 6oewzwj openmpi@1.10.2 jqhycal py-setuptools@39.2.0 zk5lhob gdbm@1.14.1 uhzmwog netlib-scalapack@2.0.2 cvldq3v openssl@1.0.2o c7pnzul python@2.7.15 - vt5xcat mpich@3.2.1 lqfhvfh netlib-scalapack@2.0.2 yoxwmgb pkgconf@1.4.2 ccruj2i readline@7.0 - xcgzqdv ncurses@6.1 jae3ilo netlib-scalapack@2.0.2 t3loxvu py-numpy@1.14.3 5ltus3a sqlite@3.23.1 + vt5xcat mpich@3.2.1 3bz5rxx netlib-scalapack@2.0.2 yoxwmgb pkgconf@1.4.2 ccruj2i readline@7.0 + xcgzqdv ncurses@6.1 z52ltyy netlib-scalapack@2.0.2 t3loxvu py-numpy@1.14.3 5ltus3a sqlite@3.23.1 ==> Do you want to proceed? [y/n] y ==> Regenerating tcl module files @@ -503,7 +503,7 @@ and regenerate the module files: .. code-block:: console - root@module-file-tutorial:/# spack tcl refresh --delete-tree + root@module-file-tutorial:/# spack module tcl refresh --delete-tree ==> You are about to regenerate tcl module files for: -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- @@ -514,11 +514,11 @@ and regenerate the module files: mkaoyhz git@2.9.4 wl6mywv lmod@7.7.29 56lbd3h mpc@1.1.0 nxhwrg7 readline@7.0 -- linux-ubuntu16.04-x86_64 / gcc@7.2.0 ------------------------- - mwamumj bzip2@1.0.6 bcikpen netlib-lapack@3.8.0 pdatzbi openblas@0.3.0 fdiryak py-scipy@1.1.0 ezuwp4p zlib@1.2.11 - h24ofjs cmake@3.11.4 d3lertf netlib-scalapack@2.0.2 2h6xmxh openmpi@1.10.2 jqhycal py-setuptools@39.2.0 + mwamumj bzip2@1.0.6 7apabqu netlib-lapack@3.8.0 pdatzbi openblas@0.3.0 fdiryak py-scipy@1.1.0 ezuwp4p zlib@1.2.11 + 6ovorxs cmake@3.12.0 6i5qsqx netlib-scalapack@2.0.2 6oewzwj openmpi@1.10.2 jqhycal py-setuptools@39.2.0 zk5lhob gdbm@1.14.1 uhzmwog netlib-scalapack@2.0.2 cvldq3v openssl@1.0.2o c7pnzul python@2.7.15 - vt5xcat mpich@3.2.1 lqfhvfh netlib-scalapack@2.0.2 yoxwmgb pkgconf@1.4.2 ccruj2i readline@7.0 - xcgzqdv ncurses@6.1 jae3ilo netlib-scalapack@2.0.2 t3loxvu py-numpy@1.14.3 5ltus3a sqlite@3.23.1 + vt5xcat mpich@3.2.1 3bz5rxx netlib-scalapack@2.0.2 yoxwmgb pkgconf@1.4.2 ccruj2i readline@7.0 + xcgzqdv ncurses@6.1 z52ltyy netlib-scalapack@2.0.2 t3loxvu py-numpy@1.14.3 5ltus3a sqlite@3.23.1 ==> Do you want to proceed? [y/n] y ==> Regenerating tcl module files @@ -526,13 +526,13 @@ and regenerate the module files: root@module-file-tutorial:/# module avail -------------------------------------------- /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64 --------------------------------------------- - bzip2-1.0.6-gcc-7.2.0-mwamumj netlib-scalapack-2.0.2-gcc-7.2.0-jae3ilo py-numpy-1.14.3-gcc-7.2.0-t3loxvu - cmake-3.11.4-gcc-7.2.0-h24ofjs netlib-scalapack-2.0.2-gcc-7.2.0-lqfhvfh py-scipy-1.1.0-gcc-7.2.0-fdiryak - gdbm-1.14.1-gcc-7.2.0-zk5lhob netlib-scalapack-2.0.2-gcc-7.2.0-uhzmwog py-setuptools-39.2.0-gcc-7.2.0-jqhycal + bzip2-1.0.6-gcc-7.2.0-mwamumj netlib-scalapack-2.0.2-gcc-7.2.0-6i5qsqx py-numpy-1.14.3-gcc-7.2.0-t3loxvu + cmake-3.12.0-gcc-7.2.0-6ovorxs netlib-scalapack-2.0.2-gcc-7.2.0-uhzmwog py-scipy-1.1.0-gcc-7.2.0-fdiryak + gdbm-1.14.1-gcc-7.2.0-zk5lhob netlib-scalapack-2.0.2-gcc-7.2.0-z52ltyy py-setuptools-39.2.0-gcc-7.2.0-jqhycal mpich-3.2.1-gcc-7.2.0-vt5xcat openblas-0.3.0-gcc-7.2.0-pdatzbi python-2.7.15-gcc-7.2.0-c7pnzul - ncurses-6.1-gcc-7.2.0-xcgzqdv openmpi-1.10.2-gcc-7.2.0-2h6xmxh readline-7.0-gcc-7.2.0-ccruj2i - netlib-lapack-3.8.0-gcc-7.2.0-bcikpen openssl-1.0.2o-gcc-7.2.0-cvldq3v sqlite-3.23.1-gcc-7.2.0-5ltus3a - netlib-scalapack-2.0.2-gcc-7.2.0-d3lertf pkgconf-1.4.2-gcc-7.2.0-yoxwmgb zlib-1.2.11-gcc-7.2.0-ezuwp4p + ncurses-6.1-gcc-7.2.0-xcgzqdv openmpi-1.10.2-gcc-7.2.0-6oewzwj readline-7.0-gcc-7.2.0-ccruj2i + netlib-lapack-3.8.0-gcc-7.2.0-7apabqu openssl-1.0.2o-gcc-7.2.0-cvldq3v sqlite-3.23.1-gcc-7.2.0-5ltus3a + netlib-scalapack-2.0.2-gcc-7.2.0-3bz5rxx pkgconf-1.4.2-gcc-7.2.0-yoxwmgb zlib-1.2.11-gcc-7.2.0-ezuwp4p Use "module spider" to find all possible modules. Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". @@ -562,7 +562,7 @@ exceptions to the blacklist rules you can use ``whitelist``: .. code-block:: console - root@module-file-tutorial:/# spack tcl refresh -y + root@module-file-tutorial:/# spack module tcl refresh -y ==> Regenerating tcl module files @@ -626,7 +626,7 @@ If you try to regenerate the module files now you will get an error: .. code-block:: console - root@module-file-tutorial:/# spack tcl refresh --delete-tree -y + root@module-file-tutorial:/# spack module tcl refresh --delete-tree -y ==> Error: Name clashes detected in module files: file: /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64/netlib-scalapack-2.0.2-gcc-7.2.0 @@ -673,14 +673,14 @@ Regenerating module files now we obtain: .. code-block:: console - root@module-file-tutorial:/# spack tcl refresh --delete-tree -y + root@module-file-tutorial:/# spack module tcl refresh --delete-tree -y ==> Regenerating tcl module files root@module-file-tutorial:/# module av -------------------------------------------- /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64 --------------------------------------------- bzip2-1.0.6-gcc-7.2.0 netlib-lapack-3.8.0-gcc-7.2.0 openmpi-1.10.2-gcc-7.2.0 python-2.7.15-gcc-7.2.0 - cmake-3.11.4-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-netlib-mpich openssl-1.0.2o-gcc-7.2.0 readline-7.0-gcc-7.2.0 + cmake-3.12.0-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-netlib-mpich openssl-1.0.2o-gcc-7.2.0 readline-7.0-gcc-7.2.0 gcc-7.2.0-gcc-5.4.0 netlib-scalapack-2.0.2-gcc-7.2.0-netlib-openmpi pkgconf-1.4.2-gcc-7.2.0 sqlite-3.23.1-gcc-7.2.0 gdbm-1.14.1-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-openblas-mpich py-numpy-1.14.3-gcc-7.2.0-openblas zlib-1.2.11-gcc-7.2.0 mpich-3.2.1-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-openblas-openmpi py-scipy-1.1.0-gcc-7.2.0-openblas @@ -720,13 +720,13 @@ The final result should look like: .. code-block:: console - root@module-file-tutorial:/# spack tcl refresh --delete-tree -y + root@module-file-tutorial:/# spack module tcl refresh --delete-tree -y ==> Regenerating tcl module files root@module-file-tutorial:/# module av -------------------------------------------- /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64 --------------------------------------------- bzip2/1.0.6-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-netlib-openmpi py-scipy/1.1.0-gcc-7.2.0-openblas - cmake/3.11.4-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-mpich py-setuptools/39.2.0-gcc-7.2.0 + cmake/3.12.0-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-mpich py-setuptools/39.2.0-gcc-7.2.0 gcc/7.2.0-gcc-5.4.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-openmpi (D) python/2.7.15-gcc-7.2.0 gdbm/1.14.1-gcc-7.2.0 openblas/0.3.0-gcc-7.2.0 readline/7.0-gcc-7.2.0 mpich/3.2.1-gcc-7.2.0 openmpi/1.10.2-gcc-7.2.0 sqlite/3.23.1-gcc-7.2.0 @@ -794,7 +794,7 @@ Regenerating the module files results in something like: .. code-block:: console :emphasize-lines: 15 - root@module-file-tutorial:/# spack tcl refresh -y + root@module-file-tutorial:/# spack module tcl refresh -y ==> Regenerating tcl module files root@module-file-tutorial:/# module show gcc @@ -870,10 +870,10 @@ This time we will be more selective and regenerate only the ``gcc`` and .. code-block:: console - root@module-file-tutorial:/# spack tcl refresh -y gcc + root@module-file-tutorial:/# spack module tcl refresh -y gcc ==> Regenerating tcl module files - root@module-file-tutorial:/# spack tcl refresh -y openmpi + root@module-file-tutorial:/# spack module tcl refresh -y openmpi ==> Regenerating tcl module files root@module-file-tutorial:/# module show gcc @@ -981,7 +981,7 @@ and regenerating the module files for every package that depends on ``python``: .. code-block:: console - root@module-file-tutorial:/# spack tcl refresh -y ^python + root@module-file-tutorial:/# spack module tcl refresh -y ^python ==> Regenerating tcl module files Now the ``py-scipy`` module will be: @@ -1165,7 +1165,7 @@ If we now regenerate the module files: .. code-block:: console - root@module-file-tutorial:/# spack lmod refresh --delete-tree -y + root@module-file-tutorial:/# spack module lmod refresh --delete-tree -y ==> Regenerating lmod module files and update ``MODULEPATH`` to point to the ``Core``: @@ -1198,7 +1198,7 @@ the ``Compiler`` part of the hierarchy: ----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 ----------------------------------------- bzip2/1.0.6 mpich/3.2.1 openblas/0.3.0 pkgconf/1.4.2 py-setuptools/39.2.0 sqlite/3.23.1 - cmake/3.11.4 ncurses/6.1 openmpi/1.10.2 py-numpy/1.14.3-openblas python/2.7.15 zlib/1.2.11 + cmake/3.12.0 ncurses/6.1 openmpi/1.10.2 py-numpy/1.14.3-openblas python/2.7.15 zlib/1.2.11 gdbm/1.14.1 netlib-lapack/3.8.0 openssl/1.0.2o py-scipy/1.1.0-openblas readline/7.0 ------------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/Core -------------------------------------------- @@ -1223,7 +1223,7 @@ either ``mpich`` or ``openmpi``. Let's start by loading ``mpich``: ----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 ----------------------------------------- bzip2/1.0.6 mpich/3.2.1 (L) openblas/0.3.0 pkgconf/1.4.2 py-setuptools/39.2.0 sqlite/3.23.1 - cmake/3.11.4 ncurses/6.1 openmpi/1.10.2 py-numpy/1.14.3-openblas python/2.7.15 zlib/1.2.11 + cmake/3.12.0 ncurses/6.1 openmpi/1.10.2 py-numpy/1.14.3-openblas python/2.7.15 zlib/1.2.11 gdbm/1.14.1 netlib-lapack/3.8.0 openssl/1.0.2o py-scipy/1.1.0-openblas readline/7.0 ------------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/Core -------------------------------------------- @@ -1357,7 +1357,7 @@ After module files have been regenerated as usual: root@module-file-tutorial:/# module purge - root@module-file-tutorial:/# spack lmod refresh --delete-tree -y + root@module-file-tutorial:/# spack module lmod refresh --delete-tree -y ==> Regenerating lmod module files we can see that now we have additional components in the hierarchy: @@ -1369,11 +1369,11 @@ we can see that now we have additional components in the hierarchy: root@module-file-tutorial:/# module avail ----------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/openblas/0.3.0-pdatzbi/gcc/7.2.0 ------------------------------ - py-numpy/1.14.3-openblas py-scipy/1.1.0-openblas + py-numpy/1.14.3 py-scipy/1.1.0 ----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 ----------------------------------------- bzip2/1.0.6 mpich/3.2.1 openblas/0.3.0 (L) pkgconf/1.4.2 readline/7.0 - cmake/3.11.4 ncurses/6.1 openmpi/1.10.2 py-setuptools/39.2.0 sqlite/3.23.1 + cmake/3.12.0 ncurses/6.1 openmpi/1.10.2 py-setuptools/39.2.0 sqlite/3.23.1 gdbm/1.14.1 netlib-lapack/3.8.0 openssl/1.0.2o python/2.7.15 zlib/1.2.11 ------------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/Core -------------------------------------------- @@ -1389,11 +1389,11 @@ we can see that now we have additional components in the hierarchy: root@module-file-tutorial:/# module load openmpi root@module-file-tutorial:/# module avail - ------------------ /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/openmpi/1.10.2-2h6xmxh/openblas/0.3.0-pdatzbi/gcc/7.2.0 ------------------ - netlib-scalapack/2.0.2-openblas + ------------------ /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/openmpi/1.10.2-6oewzwj/openblas/0.3.0-pdatzbi/gcc/7.2.0 ------------------ + netlib-scalapack/2.0.2 ----------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/openblas/0.3.0-pdatzbi/gcc/7.2.0 ------------------------------ - py-numpy/1.14.3-openblas py-scipy/1.1.0-openblas + py-numpy/1.14.3 py-scipy/1.1.0 ----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 ----------------------------------------- bzip2/1.0.6 mpich/3.2.1 openblas/0.3.0 (L) pkgconf/1.4.2 readline/7.0 @@ -1420,18 +1420,15 @@ Both ``MPI`` and ``LAPACK`` providers will now benefit from the same safety feat Due to MODULEPATH changes, the following have been reloaded: - 1) netlib-scalapack/2.0.2-openblas - - root@module-file-tutorial:/# module load netlib-lapack + 1) netlib-scalapack/2.0.2 - Lmod is automatically replacing "openblas/0.3.0" with "netlib-lapack/3.8.0". + root@module-file-tutorial:/# module load mpich + Lmod is automatically replacing "openmpi/1.10.2" with "mpich/3.2.1". - Inactive Modules: - 1) py-numpy - The following have been reloaded with a version change: - 1) netlib-scalapack/2.0.2-openblas => netlib-scalapack/2.0.2-netlib + Due to MODULEPATH changes, the following have been reloaded: + 1) netlib-scalapack/2.0.2 Because we only compiled ``py-numpy`` with ``openblas`` the module is made inactive when we switch the ``LAPACK`` provider. The user @@ -1589,7 +1586,7 @@ If we regenerate the module files one last time: .. code-block:: console - root@module-file-tutorial:/# spack lmod refresh -y netlib-scalapack + root@module-file-tutorial:/# spack module lmod refresh -y netlib-scalapack ==> Regenerating lmod module files we'll find the following at the end of each ``netlib-scalapack`` module file: diff --git a/lib/spack/docs/workflows.rst b/lib/spack/docs/workflows.rst index d3a60d9490..b3bdaf3291 100644 --- a/lib/spack/docs/workflows.rst +++ b/lib/spack/docs/workflows.rst @@ -276,11 +276,11 @@ have some drawbacks: 2. The ``spack spec`` and ``spack install`` commands use a sophisticated concretization algorithm that chooses the "best" among several options, taking into account ``packages.yaml`` file. - The ``spack load`` and ``spack tcl loads`` commands, on the + The ``spack load`` and ``spack module tcl loads`` commands, on the other hand, are not very smart: if the user-supplied spec matches - more than one installed package, then ``spack tcl loads`` will + more than one installed package, then ``spack module tcl loads`` will fail. This may change in the future. For now, the workaround is to - be more specific on any ``spack tcl loads`` lines that fail. + be more specific on any ``spack module tcl loads`` lines that fail. """""""""""""""""""""" @@ -290,7 +290,7 @@ Generated Load Scripts Another problem with using `spack load` is, it is slow; a typical user environment could take several seconds to load, and would not be appropriate to put into ``.bashrc`` directly. It is preferable to use -a series of ``spack tcl loads`` commands to pre-compute which +a series of ``spack module tcl loads`` commands to pre-compute which modules to load. These can be put in a script that is run whenever installed Spack packages change. For example: @@ -301,7 +301,7 @@ installed Spack packages change. For example: # Generate module load commands in ~/env/spackenv cat <<EOF | /bin/sh >$HOME/env/spackenv - FIND='spack tcl loads --prefix linux-SuSE11-x86_64/' + FIND='spack module tcl loads --prefix linux-SuSE11-x86_64/' \$FIND modele-utils \$FIND emacs @@ -346,14 +346,14 @@ Users may now put ``source ~/env/spackenv`` into ``.bashrc``. Some module systems put a prefix on the names of modules created by Spack. For example, that prefix is ``linux-SuSE11-x86_64/`` in the above case. If a prefix is not needed, you may omit the - ``--prefix`` flag from ``spack tcl loads``. + ``--prefix`` flag from ``spack module tcl loads``. """"""""""""""""""""""" Transitive Dependencies """"""""""""""""""""""" -In the script above, each ``spack tcl loads`` command generates a +In the script above, each ``spack module tcl loads`` command generates a *single* ``module load`` line. Transitive dependencies do not usually need to be loaded, only modules the user needs in ``$PATH``. This is because Spack builds binaries with RPATH. Spack's RPATH policy has @@ -394,13 +394,13 @@ Unfortunately, Spack's RPATH support does not work in all case. For example: In cases where RPATH support doesn't make things "just work," it can be necessary to load a module's dependencies as well as the module itself. This is done by adding the ``--dependencies`` flag to the -``spack tcl loads`` command. For example, the following line, +``spack module tcl loads`` command. For example, the following line, added to the script above, would be used to load SciPy, along with Numpy, core Python, BLAS/LAPACK and anything else needed: .. code-block:: sh - spack tcl loads --dependencies py-scipy + spack module tcl loads --dependencies py-scipy ^^^^^^^^^^^^^^ Dummy Packages @@ -630,7 +630,7 @@ environments: and extension packages. * Views and activated extensions maintain state that is semantically - equivalent to the information in a ``spack tcl loads`` script. + equivalent to the information in a ``spack module tcl loads`` script. Administrators might find things easier to maintain without the added "heavyweight" state of a view. diff --git a/lib/spack/spack/cmd/module.py b/lib/spack/spack/cmd/module.py new file mode 100644 index 0000000000..22826afdf3 --- /dev/null +++ b/lib/spack/spack/cmd/module.py @@ -0,0 +1,45 @@ +############################################################################## +# Copyright (c) 2013-2018, Lawrence Livermore National Security, LLC. +# Produced at the Lawrence Livermore National Laboratory. +# +# This file is part of Spack. +# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved. +# LLNL-CODE-647188 +# +# For details, see https://github.com/spack/spack +# Please also see the NOTICE and LICENSE files for our notice and the LGPL. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License (as +# published by the Free Software Foundation) version 2.1, February 1999. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and +# conditions of the GNU Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +############################################################################## +import spack.cmd.modules.dotkit +import spack.cmd.modules.lmod +import spack.cmd.modules.tcl + +description = "manipulate module files" +section = "environment" +level = "short" + + +_subcommands = {} + + +def setup_parser(subparser): + sp = subparser.add_subparsers(metavar='SUBCOMMAND', dest='module_type') + spack.cmd.modules.dotkit.add_command(sp, _subcommands) + spack.cmd.modules.lmod.add_command(sp, _subcommands) + spack.cmd.modules.tcl.add_command(sp, _subcommands) + + +def module(parser, args): + _subcommands[args.module_type](parser, args) diff --git a/lib/spack/spack/cmd/common/modules.py b/lib/spack/spack/cmd/modules/__init__.py index 11ee9cdb5e..784c49daa2 100644 --- a/lib/spack/spack/cmd/common/modules.py +++ b/lib/spack/spack/cmd/modules/__init__.py @@ -22,9 +22,7 @@ # License along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA ############################################################################## -"""Contains all the functions that are common to the implementation of -each module file command. -""" +"""Implementation details of the ``spack module`` command.""" import collections import os.path @@ -43,15 +41,6 @@ section = "environment" level = "short" -#: Dictionary that will be populated with the list of sub-commands -#: Each sub-command must be callable and accept 3 arguments: -#: -#: - mtype : the type of the module file -#: - specs : the list of specs to be processed -#: - args : namespace containing the parsed command line arguments -callbacks = {} - - def setup_parser(subparser): sp = subparser.add_subparsers(metavar='SUBCOMMAND', dest='subparser_name') diff --git a/lib/spack/spack/cmd/dotkit.py b/lib/spack/spack/cmd/modules/dotkit.py index 619389eb04..9c50dfa777 100644 --- a/lib/spack/spack/cmd/dotkit.py +++ b/lib/spack/spack/cmd/modules/dotkit.py @@ -22,21 +22,17 @@ # License along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA ############################################################################## -import spack.cmd.common.modules +import functools -description = "manipulate dotkit module files" -section = "environment" -level = "short" +import spack.cmd.modules -#: Type of the modules managed by this command -_module_type = 'dotkit' - - -def setup_parser(subparser): - spack.cmd.common.modules.setup_parser(subparser) +def add_command(parser, command_dict): + dotkit_parser = parser.add_parser( + 'dotkit', help='manipulate dotkit module files' + ) + spack.cmd.modules.setup_parser(dotkit_parser) -def dotkit(parser, args): - spack.cmd.common.modules.modules_cmd( - parser, args, module_type=_module_type + command_dict['dotkit'] = functools.partial( + spack.cmd.modules.modules_cmd, module_type='dotkit' ) diff --git a/lib/spack/spack/cmd/lmod.py b/lib/spack/spack/cmd/modules/lmod.py index ea3cbd7a61..b40024b114 100644 --- a/lib/spack/spack/cmd/lmod.py +++ b/lib/spack/spack/cmd/modules/lmod.py @@ -22,29 +22,33 @@ # License along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA ############################################################################## +import functools import os import llnl.util.filesystem import spack.cmd.common.arguments -import spack.cmd.common.modules +import spack.cmd.modules -description = "manipulate hierarchical module files" -section = "environment" -level = "short" -#: Type of the modules managed by this command -_module_type = 'lmod' - - -def setup_parser(subparser): - sp = spack.cmd.common.modules.setup_parser(subparser) +def add_command(parser, command_dict): + lmod_parser = parser.add_parser( + 'lmod', help='manipulate hierarchical module files' + ) + sp = spack.cmd.modules.setup_parser(lmod_parser) # Set default module file for a package - setdefault = sp.add_parser( + setdefault_parser = sp.add_parser( 'setdefault', help='set the default module file for a package' ) spack.cmd.common.arguments.add_common_arguments( - setdefault, ['constraint'] + setdefault_parser, ['constraint'] + ) + + callbacks = dict(spack.cmd.modules.callbacks.items()) + callbacks['setdefault'] = setdefault + + command_dict['lmod'] = functools.partial( + spack.cmd.modules.modules_cmd, module_type='lmod', callbacks=callbacks ) @@ -54,8 +58,8 @@ def setdefault(module_type, specs, args): # # https://lmod.readthedocs.io/en/latest/060_locating.html#marking-a-version-as-default # - spack.cmd.common.modules.one_spec_or_raise(specs) - writer = spack.modules.module_types[_module_type](specs[0]) + spack.cmd.modules.one_spec_or_raise(specs) + writer = spack.modules.module_types['lmod'](specs[0]) module_folder = os.path.dirname(writer.layout.filename) module_basename = os.path.basename(writer.layout.filename) @@ -63,13 +67,3 @@ def setdefault(module_type, specs, args): if os.path.exists('default') and os.path.islink('default'): os.remove('default') os.symlink(module_basename, 'default') - - -callbacks = dict(spack.cmd.common.modules.callbacks.items()) -callbacks['setdefault'] = setdefault - - -def lmod(parser, args): - spack.cmd.common.modules.modules_cmd( - parser, args, module_type=_module_type, callbacks=callbacks - ) diff --git a/lib/spack/spack/cmd/tcl.py b/lib/spack/spack/cmd/modules/tcl.py index 56e40e9df9..f0cf2ed13c 100644 --- a/lib/spack/spack/cmd/tcl.py +++ b/lib/spack/spack/cmd/modules/tcl.py @@ -22,21 +22,17 @@ # License along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA ############################################################################## -import spack.cmd.common.modules +import functools -description = "manipulate non-hierarchical module files" -section = "environment" -level = "short" +import spack.cmd.modules -#: Type of the modules managed by this command -_module_type = 'tcl' - - -def setup_parser(subparser): - spack.cmd.common.modules.setup_parser(subparser) +def add_command(parser, command_dict): + tcl_parser = parser.add_parser( + 'tcl', help='manipulate non-hierarchical module files' + ) + spack.cmd.modules.setup_parser(tcl_parser) -def tcl(parser, args): - spack.cmd.common.modules.modules_cmd( - parser, args, module_type=_module_type + command_dict['tcl'] = functools.partial( + spack.cmd.modules.modules_cmd, module_type='tcl' ) diff --git a/lib/spack/spack/test/cmd/dotkit.py b/lib/spack/spack/test/cmd/dotkit.py deleted file mode 100644 index a9a3ac5507..0000000000 --- a/lib/spack/spack/test/cmd/dotkit.py +++ /dev/null @@ -1,101 +0,0 @@ -############################################################################## -# Copyright (c) 2013-2018, Lawrence Livermore National Security, LLC. -# Produced at the Lawrence Livermore National Laboratory. -# -# This file is part of Spack. -# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved. -# LLNL-CODE-647188 -# -# For details, see https://github.com/spack/spack -# Please also see the NOTICE and LICENSE files for our notice and the LGPL. -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU Lesser General Public License (as -# published by the Free Software Foundation) version 2.1, February 1999. -# -# This program is distributed in the hope that it will be useful, but -# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and -# conditions of the GNU Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA -############################################################################## -import argparse -import os.path - -import pytest - -import spack.cmd.dotkit -import spack.main -import spack.modules - -dotkit = spack.main.SpackCommand('dotkit') - - -def _get_module_files(args): - specs = args.specs() - writer_cls = spack.modules.module_types['dotkit'] - return [writer_cls(spec).layout.filename for spec in specs] - - -@pytest.fixture( - params=[ - ['rm', 'doesnotexist'], # Try to remove a non existing module - ['find', 'mpileaks'], # Try to find a module with multiple matches - ['find', 'doesnotexist'], # Try to find a module with no matches - ] -) -def failure_args(request): - """A list of arguments that will cause a failure""" - return request.param - - -@pytest.fixture(scope='module') -def parser(): - """Returns the parser for the module command""" - parser = argparse.ArgumentParser() - spack.cmd.dotkit.setup_parser(parser) - return parser - -# TODO : test the --delete-tree option -# TODO : this requires having a separate directory for test modules -# TODO : add tests for loads and find to check the prompt format - - -@pytest.mark.db -@pytest.mark.usefixtures('database') -def test_exit_with_failure(database, failure_args): - with pytest.raises(spack.main.SpackCommandError): - dotkit(*failure_args) - - -@pytest.mark.db -@pytest.mark.usefixtures('database') -@pytest.mark.parametrize('cli_args', [ - ['libelf'], - ['--full-path', 'libelf'] -]) -def test_find(cli_args): - """Tests 'spack dotkit find' under a few common scenarios.""" - dotkit(*(['find'] + cli_args)) - - -@pytest.mark.db -@pytest.mark.usefixtures('database') -def test_remove_and_add_dotkit(parser): - """Tests adding and removing a dotkit module file.""" - - rm_cli_args = ['rm', '-y', 'mpileaks'] - module_files = _get_module_files(parser.parse_args(rm_cli_args)) - for item in module_files: - assert os.path.exists(item) - - dotkit(*rm_cli_args) - for item in module_files: - assert not os.path.exists(item) - - dotkit('refresh', '-y', 'mpileaks') - for item in module_files: - assert os.path.exists(item) diff --git a/lib/spack/spack/test/cmd/lmod.py b/lib/spack/spack/test/cmd/module.py index f8f9c6b6f0..ca60814147 100644 --- a/lib/spack/spack/test/cmd/lmod.py +++ b/lib/spack/spack/test/cmd/module.py @@ -28,14 +28,14 @@ import pytest import spack.main import spack.modules -import spack.spec -lmod = spack.main.SpackCommand('lmod') +module = spack.main.SpackCommand('module') -# Needed to make the fixture work -writer_cls = spack.modules.lmod.LmodModulefileWriter -# TODO : add tests for loads and find to check the prompt format +def _module_files(module_type, *specs): + specs = [spack.spec.Spec(x).concretized() for x in specs] + writer_cls = spack.modules.module_types[module_type] + return [writer_cls(spec).layout.filename for spec in specs] @pytest.fixture( @@ -50,15 +50,110 @@ def failure_args(request): return request.param -def test_exit_with_failure(database, failure_args): +@pytest.fixture( + params=['dotkit', 'tcl', 'lmod'] +) +def module_type(request): + return request.param + + +# TODO : test the --delete-tree option +# TODO : this requires having a separate directory for test modules +# TODO : add tests for loads and find to check the prompt format + +@pytest.mark.db +def test_exit_with_failure(database, module_type, failure_args): with pytest.raises(spack.main.SpackCommandError): - lmod(*failure_args) + module(module_type, *failure_args) + + +@pytest.mark.db +def test_remove_and_add(database, module_type): + """Tests adding and removing a tcl module file.""" + + if module_type == 'lmod': + # TODO: Testing this with lmod requires mocking + # TODO: the core compilers + return + + rm_cli_args = ['rm', '-y', 'mpileaks'] + module_files = _module_files(module_type, 'mpileaks') + for item in module_files: + assert os.path.exists(item) + + module(module_type, *rm_cli_args) + for item in module_files: + assert not os.path.exists(item) + + module(module_type, 'refresh', '-y', 'mpileaks') + for item in module_files: + assert os.path.exists(item) + + +@pytest.mark.db +@pytest.mark.parametrize('cli_args', [ + ['libelf'], + ['--full-path', 'libelf'] +]) +def test_find(database, cli_args, module_type): + if module_type == 'lmod': + # TODO: Testing this with lmod requires mocking + # TODO: the core compilers + return + + module(module_type, *(['find'] + cli_args)) + + +@pytest.mark.db +@pytest.mark.usefixtures('database') +@pytest.mark.regression('2215') +def test_find_fails_on_multiple_matches(): + # As we installed multiple versions of mpileaks, the command will + # fail because of multiple matches + out = module('tcl', 'find', 'mpileaks', fail_on_error=False) + assert module.returncode == 1 + assert 'matches multiple packages' in out + + # Passing multiple packages from the command line also results in the + # same failure + out = module( + 'tcl', 'find', 'mpileaks ^mpich', 'libelf', fail_on_error=False + ) + assert module.returncode == 1 + assert 'matches multiple packages' in out +@pytest.mark.db +@pytest.mark.usefixtures('database') +@pytest.mark.regression('2570') +def test_find_fails_on_non_existing_packages(): + # Another way the command might fail is if the package does not exist + out = module('tcl', 'find', 'doesnotexist', fail_on_error=False) + assert module.returncode == 1 + assert 'matches no package' in out + + +@pytest.mark.db +@pytest.mark.usefixtures('database') +def test_find_recursive(): + # If we call find without options it should return only one module + out = module('tcl', 'find', 'mpileaks ^zmpi') + assert len(out.split()) == 1 + + # If instead we call it with the recursive option the length should + # be greater + out = module('tcl', 'find', '-r', 'mpileaks ^zmpi') + assert len(out.split()) > 1 + + +# Needed to make the 'module_configuration' fixture below work +writer_cls = spack.modules.lmod.LmodModulefileWriter + + +@pytest.mark.db def test_setdefault_command( mutable_database, module_configuration ): - module_configuration('autoload_direct') # Install two different versions of a package @@ -73,7 +168,7 @@ def test_setdefault_command( } # Create two module files for the same software - lmod('refresh', '-y', '--delete-tree', preferred, other_spec) + module('lmod', 'refresh', '-y', '--delete-tree', preferred, other_spec) # Assert initial directory state: no link and all module files present link_name = os.path.join( @@ -85,7 +180,7 @@ def test_setdefault_command( assert not os.path.exists(link_name) # Set the default to be the other spec - lmod('setdefault', other_spec) + module('lmod', 'setdefault', other_spec) # Check that a link named 'default' exists, and points to the right file for k in preferred, other_spec: @@ -94,7 +189,7 @@ def test_setdefault_command( assert os.path.realpath(link_name) == writers[other_spec].layout.filename # Reset the default to be the preferred spec - lmod('setdefault', preferred) + module('lmod', 'setdefault', preferred) # Check that a link named 'default' exists, and points to the right file for k in preferred, other_spec: diff --git a/lib/spack/spack/test/cmd/tcl.py b/lib/spack/spack/test/cmd/tcl.py deleted file mode 100644 index d5deffce6b..0000000000 --- a/lib/spack/spack/test/cmd/tcl.py +++ /dev/null @@ -1,135 +0,0 @@ -############################################################################## -# Copyright (c) 2013-2017, Lawrence Livermore National Security, LLC. -# Produced at the Lawrence Livermore National Laboratory. -# -# This file is part of Spack. -# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved. -# LLNL-CODE-647188 -# -# For details, see https://github.com/spack/spack -# Please also see the NOTICE and LICENSE files for our notice and the LGPL. -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU Lesser General Public License (as -# published by the Free Software Foundation) version 2.1, February 1999. -# -# This program is distributed in the hope that it will be useful, but -# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and -# conditions of the GNU Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this program; if not, write to the Free Software -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA -############################################################################## -import argparse -import os.path - -import pytest - -import spack.cmd.tcl -import spack.main -import spack.modules - -tcl = spack.main.SpackCommand('tcl') - - -def _get_module_files(args): - specs = args.specs() - writer_cls = spack.modules.module_types['tcl'] - return [writer_cls(spec).layout.filename for spec in specs] - - -@pytest.fixture( - params=[ - ['rm', 'doesnotexist'], # Try to remove a non existing module - ['find', 'mpileaks'], # Try to find a module with multiple matches - ['find', 'doesnotexist'], # Try to find a module with no matches - ] -) -def failure_args(request): - """A list of arguments that will cause a failure""" - return request.param - - -@pytest.fixture(scope='module') -def parser(): - """Returns the parser for the module command""" - parser = argparse.ArgumentParser() - spack.cmd.tcl.setup_parser(parser) - return parser - -# TODO : test the --delete-tree option -# TODO : this requires having a separate directory for test modules -# TODO : add tests for loads and find to check the prompt format - - -def test_exit_with_failure(database, failure_args): - with pytest.raises(spack.main.SpackCommandError): - tcl(*failure_args) - - -def test_remove_and_add_tcl(database, parser): - """Tests adding and removing a tcl module file.""" - - rm_cli_args = ['rm', '-y', 'mpileaks'] - module_files = _get_module_files(parser.parse_args(rm_cli_args)) - for item in module_files: - assert os.path.exists(item) - - tcl(*rm_cli_args) - for item in module_files: - assert not os.path.exists(item) - - tcl('refresh', '-y', 'mpileaks') - for item in module_files: - assert os.path.exists(item) - - -@pytest.mark.parametrize('cli_args', [ - ['libelf'], - ['--full-path', 'libelf'] -]) -def test_find(database, cli_args): - """Tests 'spack tcl find' under a few common scenarios.""" - tcl(*(['find'] + cli_args)) - - -@pytest.mark.db -@pytest.mark.usefixtures('database') -@pytest.mark.regression('2215') -def test_find_fails_on_multiple_matches(): - # As we installed multiple versions of mpileaks, the command will - # fail because of multiple matches - out = tcl('find', 'mpileaks', fail_on_error=False) - assert tcl.returncode == 1 - assert 'matches multiple packages' in out - - # Passing multiple packages from the command line also results in the - # same failure - out = tcl('find', 'mpileaks ^mpich', 'libelf', fail_on_error=False) - assert tcl.returncode == 1 - assert 'matches multiple packages' in out - - -@pytest.mark.db -@pytest.mark.usefixtures('database') -@pytest.mark.regression('2570') -def test_find_fails_on_non_existing_packages(): - # Another way the command might fail is if the package does not exist - out = tcl('find', 'doesnotexist', fail_on_error=False) - assert tcl.returncode == 1 - assert 'matches no package' in out - - -@pytest.mark.db -@pytest.mark.usefixtures('database') -def test_find_recursive(): - # If we call find without options it should return only one module - out = tcl('find', 'mpileaks ^zmpi') - assert len(out.split()) == 1 - - # If instead we call it with the recursive option the length should - # be greater - out = tcl('find', '-r', 'mpileaks ^zmpi') - assert len(out.split()) > 1 |