diff options
author | Todd Gamblin <tgamblin@llnl.gov> | 2016-11-13 13:29:28 -0700 |
---|---|---|
committer | Todd Gamblin <tgamblin@llnl.gov> | 2016-11-14 18:08:21 -0700 |
commit | bdd1069a9ee8ed5ddc5fee5006782543c04002b3 (patch) | |
tree | 74c89e1bce6dd9cdb0febed228a326cc0a494c2b /lib/spack/docs/tutorial_sc16_modules.rst | |
parent | b13b253ced6bf501c4b74608bc8426d51c2b2356 (diff) | |
download | spack-bdd1069a9ee8ed5ddc5fee5006782543c04002b3.tar.gz spack-bdd1069a9ee8ed5ddc5fee5006782543c04002b3.tar.bz2 spack-bdd1069a9ee8ed5ddc5fee5006782543c04002b3.tar.xz spack-bdd1069a9ee8ed5ddc5fee5006782543c04002b3.zip |
Add SC16 tutorial to the Spack docs.
Diffstat (limited to 'lib/spack/docs/tutorial_sc16_modules.rst')
-rw-r--r-- | lib/spack/docs/tutorial_sc16_modules.rst | 982 |
1 files changed, 982 insertions, 0 deletions
diff --git a/lib/spack/docs/tutorial_sc16_modules.rst b/lib/spack/docs/tutorial_sc16_modules.rst new file mode 100644 index 0000000000..1e786674bb --- /dev/null +++ b/lib/spack/docs/tutorial_sc16_modules.rst @@ -0,0 +1,982 @@ +.. _modules-tutorial: + +============================= +Module Configuration Tutorial +============================= + +This tutorial will guide you through the customization of both +content and naming of module files generated by Spack. + +Starting from the default Spack settings you will add an increasing +number of directives to the ``modules.yaml`` configuration file to +satisfy a number of constraints that mimic those that you may encounter +in a typical production environment at HPC sites. + +Even though the focus will be for the most part on customizing +TCL non-hierarchical module files, everything +you'll see applies also to other kinds of module files generated by Spack. + +The generation of Lua hierarchical +module files will be addressed at the end of the tutorial, +and you'll see that with minor modifications +to an existing ``modules.yaml`` written for TCL +non-hierarchical modules you'll get almost +for free the possibility to try a hierarchical layout. + +Let's start! + +.. _module_file_tutorial_prerequisites: + +------------- +Prerequisites +------------- + +Before proceeding further ensure: + +- you have LMod or Environment Modules available +- have :ref:`shell support <shell_support>` activated in Spack + +If you need to install Lmod or Environment module you can refer +to the documentation :ref:`here <InstallEnvironmentModules>`. + + +^^^^^^^^^^^^^^^^^^ +Add a new compiler +^^^^^^^^^^^^^^^^^^ + +Spack automatically scans the environment to search for available +compilers on first use. On a Ubuntu 14.04 a fresh clone will show +something like this: + +.. code-block:: console + + $ uname -a + Linux nuvolari 4.4.0-45-generic #66~14.04.1-Ubuntu SMP Wed Oct 19 15:05:38 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux + + $ spack compilers + ==> Available compilers + -- gcc ---------------------------------------------------------- + gcc@4.8 + +For the purpose of building a limited set of packages with some features +that will help showcasing the capabilities of +module customization the first thing we need is to build a new compiler: + +.. code-block:: console + + $ spack install gcc@6.2.0 + # ... + # Wait a long time + # ... + +Then we can use shell support for modules to add it to the list of known compilers: + +.. code-block:: console + + # The name of the generated module may vary + $ module load gcc-6.2.0-gcc-4.8-twd5nqg + + $ spack compiler add + ==> Added 1 new compiler to /home/mculpo/.spack/linux/compilers.yaml + gcc@6.2.0 + + $ spack compilers + ==> Available compilers + -- gcc ---------------------------------------------------------- + gcc@6.2.0 gcc@4.8 + +Note that the final 7 digits hash at the end of the generated module may vary depending +on architecture or package version. + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Build software that will be used in the tutorial +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Next you should install a few modules that will be used in the tutorial: + +.. code-block:: console + + $ spack install netlib-scalapack ^openmpi ^openblas + # ... + +The packages you need to install are: + +- ``netlib-scalapack ^openmpi ^openblas`` +- ``netlib-scalapack ^mpich ^openblas`` +- ``netlib-scalapack ^openmpi ^netlib-lapack`` +- ``netlib-scalapack ^mpich ^netlib-lapack`` +- ``py-scipy ^openblas`` + +In the end your environment should look something like: + +.. code-block:: console + + $ module avail + + ------------------------------------------------------------------------ /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------ + binutils-2.27-gcc-4.8-dz3xevw libpciaccess-0.13.4-gcc-6.2.0-eo2siet lzo-2.09-gcc-6.2.0-jcngz72 netlib-scalapack-2.0.2-gcc-6.2.0-wnimqhw python-2.7.12-gcc-6.2.0-qu7rc5p + bzip2-1.0.6-gcc-6.2.0-csoc2mq libsigsegv-2.10-gcc-4.8-avb6azw m4-1.4.17-gcc-4.8-iggewke netlib-scalapack-2.0.2-gcc-6.2.0-wojunhq sqlite-3.8.5-gcc-6.2.0-td3zfe7 + cmake-3.5.2-gcc-6.2.0-6poypqg libsigsegv-2.10-gcc-6.2.0-g3qpmbi m4-1.4.17-gcc-6.2.0-lhgqa6s nettle-3.2-gcc-6.2.0-djdthlh tcl-8.6.5-gcc-4.8-atddxu7 + curl-7.50.3-gcc-6.2.0-2ffacqm libtool-2.4.6-gcc-6.2.0-kiepac6 mpc-1.0.3-gcc-4.8-lylv7lk openblas-0.2.19-gcc-6.2.0-js33umc util-macros-1.19.0-gcc-6.2.0-uoukuqk + expat-2.2.0-gcc-6.2.0-bxqnjar libxml2-2.9.4-gcc-6.2.0-3k4ykbe mpfr-3.1.4-gcc-4.8-bldfx3w openmpi-2.0.1-gcc-6.2.0-s3qbtby xz-5.2.2-gcc-6.2.0-t5lk6in + gcc-6.2.0-gcc-4.8-twd5nqg lmod-6.4.5-gcc-4.8-7v7bh7b mpich-3.2-gcc-6.2.0-5n5xoep openssl-1.0.2j-gcc-6.2.0-hibnfda zlib-1.2.8-gcc-4.8-bds4ies + gmp-6.1.1-gcc-4.8-uq52e2n lua-5.3.2-gcc-4.8-xozf2hx ncurses-6.0-gcc-4.8-u62fit4 pkg-config-0.29.1-gcc-6.2.0-rslsgcs zlib-1.2.8-gcc-6.2.0-asydrba + gmp-6.1.1-gcc-6.2.0-3cfh3hi lua-luafilesystem-1_6_3-gcc-4.8-sbzejlz ncurses-6.0-gcc-6.2.0-7tb426s py-nose-1.3.7-gcc-6.2.0-4gl5c42 + hwloc-1.11.4-gcc-6.2.0-3ostwel lua-luaposix-33.4.0-gcc-4.8-xf7y2p5 netlib-lapack-3.6.1-gcc-6.2.0-mirer2l py-numpy-1.11.1-gcc-6.2.0-i3rpk4e + isl-0.14-gcc-4.8-cq73t5m lz4-131-gcc-6.2.0-cagoem4 netlib-scalapack-2.0.2-gcc-6.2.0-6bqlxqy py-scipy-0.18.1-gcc-6.2.0-e6uljfi + libarchive-3.2.1-gcc-6.2.0-2b54aos lzma-4.32.7-gcc-6.2.0-sfmeynw netlib-scalapack-2.0.2-gcc-6.2.0-hpqb3dp py-setuptools-25.2.0-gcc-6.2.0-hkqauaa + +------------------------------------------------ +Filter unwanted modifications to the environment +------------------------------------------------ + +The non-hierarchical TCL module files that have been generated so far +follow the default rules for module generation, which are given +:ref:`here <modules-yaml>` in the reference part of the manual. Taking a +look at the ``gcc`` module you'll see something like: + +.. code-block:: console + + $ module show gcc-6.2.0-gcc-4.8-twd5nqg + --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc-6.2.0-gcc-4.8-twd5nqg: + --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + whatis("gcc @6.2.0 ") + prepend_path("PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin") + prepend_path("CMAKE_PREFIX_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/") + prepend_path("MANPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man") + prepend_path("PKG_CONFIG_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig") + prepend_path("LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64") + prepend_path("LD_LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64") + prepend_path("CPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/include") + help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C, + Fortran, and Java. + ]]) + +As expected, a few environment variables representing paths will be modified +by the modules according to the default prefix inspection rules. + +Consider now the case that your site has decided that e.g. ``CPATH`` and +``LIBRARY_PATH`` modifications should not be present in module files. What you can +do to abide by the rules is to create a configuration file ``~/.spack/modules.yaml`` +with the following content: + +.. code-block:: yaml + + modules: + tcl: + all: + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + +Next you should regenerate all the module files: + +.. code-block:: console + + $ spack module refresh --module-type tcl + ==> You are about to regenerate tcl module files for: + + -- linux-Ubuntu14-x86_64 / gcc@4.8 ------------------------------ + dz3xevw binutils@2.27 uq52e2n gmp@6.1.1 avb6azw libsigsegv@2.10 xozf2hx lua@5.3.2 xf7y2p5 lua-luaposix@33.4.0 lylv7lk mpc@1.0.3 u62fit4 ncurses@6.0 bds4ies zlib@1.2.8 + twd5nqg gcc@6.2.0 cq73t5m isl@0.14 7v7bh7b lmod@6.4.5 sbzejlz lua-luafilesystem@1_6_3 iggewke m4@1.4.17 bldfx3w mpfr@3.1.4 atddxu7 tcl@8.6.5 + + ... + + ==> Do you want to proceed ? [y/n] + y + ==> Regenerating tcl module files + +If you take a look now at the module for ``gcc`` you'll see that the unwanted +paths have disappeared: + +.. code-block:: console + + $ module show gcc-6.2.0-gcc-4.8-twd5nqg + --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc-6.2.0-gcc-4.8-twd5nqg: + --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + whatis("gcc @6.2.0 ") + prepend_path("PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin") + prepend_path("CMAKE_PREFIX_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/") + prepend_path("MANPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man") + prepend_path("PKG_CONFIG_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig") + prepend_path("LD_LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64") + help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C, + Fortran, and Java. + ]]) + +---------------------------------------------- +Prevent some module files from being generated +---------------------------------------------- + +Another common request at many sites is to avoid exposing software that +is only needed as an intermediate step when building a newer stack. +Let's try to prevent the generation of +module files for anything that is compiled with ``gcc@4.8`` (the OS provided compiler). + +To do this you should add a ``blacklist`` keyword to the configuration file: + +.. code-block:: yaml + :emphasize-lines: 3,4 + + modules: + tcl: + blacklist: + - '%gcc@4.8' + all: + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + +and regenerate the module files: + +.. code-block:: console + + $ spack module refresh --module-type tcl --delete-tree + ==> You are about to regenerate tcl module files for: + + -- linux-Ubuntu14-x86_64 / gcc@4.8 ------------------------------ + dz3xevw binutils@2.27 uq52e2n gmp@6.1.1 avb6azw libsigsegv@2.10 xozf2hx lua@5.3.2 xf7y2p5 lua-luaposix@33.4.0 lylv7lk mpc@1.0.3 u62fit4 ncurses@6.0 bds4ies zlib@1.2.8 + twd5nqg gcc@6.2.0 cq73t5m isl@0.14 7v7bh7b lmod@6.4.5 sbzejlz lua-luafilesystem@1_6_3 iggewke m4@1.4.17 bldfx3w mpfr@3.1.4 atddxu7 tcl@8.6.5 + + -- linux-Ubuntu14-x86_64 / gcc@6.2.0 ---------------------------- + csoc2mq bzip2@1.0.6 2b54aos libarchive@3.2.1 sfmeynw lzma@4.32.7 wnimqhw netlib-scalapack@2.0.2 s3qbtby openmpi@2.0.1 hkqauaa py-setuptools@25.2.0 + 6poypqg cmake@3.5.2 eo2siet libpciaccess@0.13.4 jcngz72 lzo@2.09 6bqlxqy netlib-scalapack@2.0.2 hibnfda openssl@1.0.2j qu7rc5p python@2.7.12 + 2ffacqm curl@7.50.3 g3qpmbi libsigsegv@2.10 lhgqa6s m4@1.4.17 wojunhq netlib-scalapack@2.0.2 rslsgcs pkg-config@0.29.1 td3zfe7 sqlite@3.8.5 + bxqnjar expat@2.2.0 kiepac6 libtool@2.4.6 5n5xoep mpich@3.2 hpqb3dp netlib-scalapack@2.0.2 4gl5c42 py-nose@1.3.7 uoukuqk util-macros@1.19.0 + 3cfh3hi gmp@6.1.1 3k4ykbe libxml2@2.9.4 7tb426s ncurses@6.0 djdthlh nettle@3.2 i3rpk4e py-numpy@1.11.1 t5lk6in xz@5.2.2 + 3ostwel hwloc@1.11.4 cagoem4 lz4@131 mirer2l netlib-lapack@3.6.1 js33umc openblas@0.2.19 e6uljfi py-scipy@0.18.1 asydrba zlib@1.2.8 + + ==> Do you want to proceed ? [y/n] + y + + $ module avail + + ------------------------------------------------------------------------ /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------ + bzip2-1.0.6-gcc-6.2.0-csoc2mq libsigsegv-2.10-gcc-6.2.0-g3qpmbi ncurses-6.0-gcc-6.2.0-7tb426s openmpi-2.0.1-gcc-6.2.0-s3qbtby sqlite-3.8.5-gcc-6.2.0-td3zfe7 + cmake-3.5.2-gcc-6.2.0-6poypqg libtool-2.4.6-gcc-6.2.0-kiepac6 netlib-lapack-3.6.1-gcc-6.2.0-mirer2l openssl-1.0.2j-gcc-6.2.0-hibnfda util-macros-1.19.0-gcc-6.2.0-uoukuqk + curl-7.50.3-gcc-6.2.0-2ffacqm libxml2-2.9.4-gcc-6.2.0-3k4ykbe netlib-scalapack-2.0.2-gcc-6.2.0-6bqlxqy pkg-config-0.29.1-gcc-6.2.0-rslsgcs xz-5.2.2-gcc-6.2.0-t5lk6in + expat-2.2.0-gcc-6.2.0-bxqnjar lz4-131-gcc-6.2.0-cagoem4 netlib-scalapack-2.0.2-gcc-6.2.0-hpqb3dp py-nose-1.3.7-gcc-6.2.0-4gl5c42 zlib-1.2.8-gcc-6.2.0-asydrba + gmp-6.1.1-gcc-6.2.0-3cfh3hi lzma-4.32.7-gcc-6.2.0-sfmeynw netlib-scalapack-2.0.2-gcc-6.2.0-wnimqhw py-numpy-1.11.1-gcc-6.2.0-i3rpk4e + hwloc-1.11.4-gcc-6.2.0-3ostwel lzo-2.09-gcc-6.2.0-jcngz72 netlib-scalapack-2.0.2-gcc-6.2.0-wojunhq py-scipy-0.18.1-gcc-6.2.0-e6uljfi + libarchive-3.2.1-gcc-6.2.0-2b54aos m4-1.4.17-gcc-6.2.0-lhgqa6s nettle-3.2-gcc-6.2.0-djdthlh py-setuptools-25.2.0-gcc-6.2.0-hkqauaa + libpciaccess-0.13.4-gcc-6.2.0-eo2siet mpich-3.2-gcc-6.2.0-5n5xoep openblas-0.2.19-gcc-6.2.0-js33umc python-2.7.12-gcc-6.2.0-qu7rc5p + +This time it is convenient to pass the option ``--delete-tree`` to the command that +regenerates the module files to instruct it to delete the existing tree and regenerate +a new one instead of overwriting the files in the existing directory. + +If you pay careful attention you'll see though that we went too far in blacklisting modules: +the module for ``gcc@6.2.0`` disappeared as it was bootstrapped with ``gcc@4.8``. To specify +exceptions to the blacklist rules you can use ``whitelist``: + +.. code-block:: yaml + :emphasize-lines: 3,4 + + modules: + tcl: + whitelist: + - gcc + blacklist: + - '%gcc@4.8' + all: + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + +``whitelist`` rules always have precedence over ``blacklist`` rules. If you regenerate the modules again: + +.. code-block:: console + + $ spack module refresh --module-type tcl -y + +you'll see that now the module for ``gcc@6.2.0`` has reappeared: + +.. code-block:: console + + $ module avail gcc-6.2.0-gcc-4.8-twd5nqg + + ------------------------------------------------------------------------ /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------ + gcc-6.2.0-gcc-4.8-twd5nqg + +------------------------- +Change module file naming +------------------------- + +The next step in making module files more user-friendly is to +improve their naming scheme. +To reduce the length of the hash or remove it altogether you can +use the ``hash_length`` keyword in the configuration file: + +.. TODO: give reasons to remove hashes if they are not evident enough? + +.. code-block:: yaml + :emphasize-lines: 3 + + modules: + tcl: + hash_length: 0 + whitelist: + - gcc + blacklist: + - '%gcc@4.8' + all: + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + +If you try to regenerate the module files now you will get an error: + +.. code-block:: console + + $ spack module refresh --module-type tcl --delete-tree -y + ==> Error: Name clashes detected in module files: + + file : /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/netlib-scalapack-2.0.2-gcc-6.2.0 + spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64 + spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64 + spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64 + spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64 + + ==> Error: Operation aborted + +.. note:: + We try to check for errors upfront! + In Spack we check for errors upfront whenever possible, so don't worry about your module files: + as a name clash was detected nothing has been changed on disk. + +The problem here is that without +the hashes the four different flavors of ``netlib-scalapack`` map to the same module file +name. We have the possibility to add suffixes to differentiate them: + +.. code-block:: yaml + :emphasize-lines: 9-11,14-17 + + modules: + tcl: + hash_length: 0 + whitelist: + - gcc + blacklist: + - '%gcc@4.8' + all: + suffixes: + '^openblas': openblas + '^netlib-lapack': netlib + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + netlib-scalapack: + suffixes: + '^openmpi': openmpi + '^mpich': mpich + +As you can see it is possible to specify rules that applies only to a +restricted set of packages using :ref:`anonymous specs <anonymous_specs>`. +Regenerating module files now we obtain: + +.. code-block:: console + + $ spack module refresh --module-type tcl --delete-tree -y + ==> Regenerating tcl module files + $ module avail + + ------------------------------------------------------------------------ /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------ + bzip2-1.0.6-gcc-6.2.0 libpciaccess-0.13.4-gcc-6.2.0 mpich-3.2-gcc-6.2.0 openblas-0.2.19-gcc-6.2.0 python-2.7.12-gcc-6.2.0 + cmake-3.5.2-gcc-6.2.0 libsigsegv-2.10-gcc-6.2.0 ncurses-6.0-gcc-6.2.0 openmpi-2.0.1-gcc-6.2.0 sqlite-3.8.5-gcc-6.2.0 + curl-7.50.3-gcc-6.2.0 libtool-2.4.6-gcc-6.2.0 netlib-lapack-3.6.1-gcc-6.2.0 openssl-1.0.2j-gcc-6.2.0 util-macros-1.19.0-gcc-6.2.0 + expat-2.2.0-gcc-6.2.0 libxml2-2.9.4-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-netlib-mpich pkg-config-0.29.1-gcc-6.2.0 xz-5.2.2-gcc-6.2.0 + gcc-6.2.0-gcc-4.8 lz4-131-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-netlib-openmpi py-nose-1.3.7-gcc-6.2.0 zlib-1.2.8-gcc-6.2.0 + gmp-6.1.1-gcc-6.2.0 lzma-4.32.7-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-openblas-mpich py-numpy-1.11.1-gcc-6.2.0-openblas + hwloc-1.11.4-gcc-6.2.0 lzo-2.09-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-openblas-openmpi py-scipy-0.18.1-gcc-6.2.0-openblas + libarchive-3.2.1-gcc-6.2.0 m4-1.4.17-gcc-6.2.0 nettle-3.2-gcc-6.2.0 py-setuptools-25.2.0-gcc-6.2.0 + +Finally we can set a ``naming_scheme`` to prevent users from loading +modules that refer to different flavors of the same library/application: + +.. code-block:: yaml + :emphasize-lines: 4,10,11 + + modules: + tcl: + hash_length: 0 + naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}' + whitelist: + - gcc + blacklist: + - '%gcc@4.8' + all: + conflict: + - '${PACKAGE}' + suffixes: + '^openblas': openblas + '^netlib-lapack': netlib + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + netlib-scalapack: + suffixes: + '^openmpi': openmpi + '^mpich': mpich + +The final result should look like: + +.. code-block:: console + + $ module avail + + ------------------------------------------------------------------------ /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------ + bzip2/1.0.6-gcc-6.2.0 libpciaccess/0.13.4-gcc-6.2.0 mpich/3.2-gcc-6.2.0 openblas/0.2.19-gcc-6.2.0 python/2.7.12-gcc-6.2.0 + cmake/3.5.2-gcc-6.2.0 libsigsegv/2.10-gcc-6.2.0 ncurses/6.0-gcc-6.2.0 openmpi/2.0.1-gcc-6.2.0 sqlite/3.8.5-gcc-6.2.0 + curl/7.50.3-gcc-6.2.0 libtool/2.4.6-gcc-6.2.0 netlib-lapack/3.6.1-gcc-6.2.0 openssl/1.0.2j-gcc-6.2.0 util-macros/1.19.0-gcc-6.2.0 + expat/2.2.0-gcc-6.2.0 libxml2/2.9.4-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-netlib-mpich pkg-config/0.29.1-gcc-6.2.0 xz/5.2.2-gcc-6.2.0 + gcc/6.2.0-gcc-4.8 lz4/131-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-netlib-openmpi py-nose/1.3.7-gcc-6.2.0 zlib/1.2.8-gcc-6.2.0 + gmp/6.1.1-gcc-6.2.0 lzma/4.32.7-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-openblas-mpich py-numpy/1.11.1-gcc-6.2.0-openblas + hwloc/1.11.4-gcc-6.2.0 lzo/2.09-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-openblas-openmpi (D) py-scipy/0.18.1-gcc-6.2.0-openblas + libarchive/3.2.1-gcc-6.2.0 m4/1.4.17-gcc-6.2.0 nettle/3.2-gcc-6.2.0 py-setuptools/25.2.0-gcc-6.2.0 + +.. note:: + TCL specific directive + The directives ``naming_scheme`` and ``conflict`` are TCL specific and do not apply + to the ``dotkit`` or ``lmod`` sections in the configuration file. + +------------------------------------ +Add custom environment modifications +------------------------------------ + +At many sites it is customary to set an environment variable in a +package's module file that points to the folder in which the package +is installed. You can achieve this with Spack by adding an +``environment`` directive to the configuration file: + +.. code-block:: yaml + :emphasize-lines: 17-19 + + modules: + tcl: + hash_length: 0 + naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}' + whitelist: + - gcc + blacklist: + - '%gcc@4.8' + all: + conflict: + - '${PACKAGE}' + suffixes: + '^openblas': openblas + '^netlib-lapack': netlib + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + environment: + set: + '${PACKAGE}_ROOT': '${PREFIX}' + netlib-scalapack: + suffixes: + '^openmpi': openmpi + '^mpich': mpich + +There are many variable tokens available to use in the ``environment`` +and ``naming_scheme`` directives, such as ``${PACKAGE}``, +``${VERSION}``, etc. (see the :meth:`~spack.spec.Spec.format` API +documentation for the complete list). + +Regenerating the module files should result in something like: + +.. code-block:: console + :emphasize-lines: 14 + + $ spack module refresh -y --module-type tcl + ==> Regenerating tcl module files + + $ module show gcc + --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc/6.2.0-gcc-4.8: + --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + whatis("gcc @6.2.0 ") + prepend_path("PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin") + prepend_path("CMAKE_PREFIX_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/") + prepend_path("MANPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man") + prepend_path("PKG_CONFIG_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig") + prepend_path("LD_LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64") + setenv("GCC_ROOT","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u") + conflict("gcc") + help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C, + Fortran, and Java. + ]]) + +As you see the ``gcc`` module has the environment variable ``GCC_ROOT`` set. + +Sometimes it's also useful to apply environment modifications selectively and target +only certain packages. You can, for instance set the common variables ``CC``, ``CXX``, +etc. in the ``gcc`` module file and apply other custom modifications to the +``openmpi`` modules as follows: + +.. code-block:: yaml + :emphasize-lines: 20-32 + + modules: + tcl: + hash_length: 0 + naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}' + whitelist: + - gcc + blacklist: + - '%gcc@4.8' + all: + conflict: + - '${PACKAGE}' + suffixes: + '^openblas': openblas + '^netlib-lapack': netlib + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + environment: + set: + '${PACKAGE}_ROOT': '${PREFIX}' + gcc: + environment: + set: + CC: gcc + CXX: g++ + FC: gfortran + F90: gfortran + F77: gfortran + openmpi: + environment: + set: + SLURM_MPI_TYPE: pmi2 + OMPI_MCA_btl_openib_warn_default_gid_prefix: '0' + netlib-scalapack: + suffixes: + '^openmpi': openmpi + '^mpich': mpich + +This time we will be more selective and regenerate only the ``gcc`` and +``openmpi`` module files: + +.. code-block:: console + + $ spack module refresh -y --module-type tcl gcc + ==> Regenerating tcl module files + + $ spack module refresh -y --module-type tcl openmpi + ==> Regenerating tcl module files + + $ module show gcc + --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc/6.2.0-gcc-4.8: + --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + whatis("gcc @6.2.0 ") + prepend_path("PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin") + prepend_path("CMAKE_PREFIX_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/") + prepend_path("MANPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man") + prepend_path("PKG_CONFIG_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig") + prepend_path("LD_LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64") + setenv("GCC_ROOT","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u") + setenv("CC","gcc") + setenv("CXX","g++") + setenv("F90","gfortran") + setenv("FC","gfortran") + setenv("F77","gfortran") + conflict("gcc") + help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C, + Fortran, and Java. + ]]) + + $ module show openmpi + --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/openmpi/2.0.1-gcc-6.2.0: + --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + whatis("openmpi @2.0.1 ") + prepend_path("PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/bin") + prepend_path("CMAKE_PREFIX_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/") + prepend_path("LD_LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/lib") + prepend_path("PKG_CONFIG_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/lib/pkgconfig") + prepend_path("MANPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/share/man") + setenv("SLURM_MPI_TYPE","pmi2") + setenv("OMPI_MCA_BTL_OPENIB_WARN_DEFAULT_GID_PREFIX","0") + setenv("OPENMPI_ROOT","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w") + conflict("openmpi") + help([[The Open MPI Project is an open source Message Passing Interface + implementation that is developed and maintained by a consortium of + academic, research, and industry partners. Open MPI is therefore able to + combine the expertise, technologies, and resources from all across the + High Performance Computing community in order to build the best MPI + library available. Open MPI offers advantages for system and software + vendors, application developers and computer science researchers. + ]]) + + +--------------------- +Autoload dependencies +--------------------- + +Spack can also generate module files that contain code to load the +dependencies automatically. You can, for instance generate python +modules that load their dependencies by adding the ``autoload`` +directive and assigning it the value ``direct``: + +.. code-block:: yaml + :emphasize-lines: 37,38 + + modules: + tcl: + hash_length: 0 + naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}' + whitelist: + - gcc + blacklist: + - '%gcc@4.8' + all: + conflict: + - '${PACKAGE}' + suffixes: + '^openblas': openblas + '^netlib-lapack': netlib + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + environment: + set: + '${PACKAGE}_ROOT': '${PREFIX}' + gcc: + environment: + set: + CC: gcc + CXX: g++ + FC: gfortran + F90: gfortran + F77: gfortran + openmpi: + environment: + set: + SLURM_MPI_TYPE: pmi2 + OMPI_MCA_btl_openib_warn_default_gid_prefix: '0' + netlib-scalapack: + suffixes: + '^openmpi': openmpi + '^mpich': mpich + ^python: + autoload: 'direct' + +and regenerating the module files for every package that depends on ``python``: + +.. code-block:: console + + $ spack module refresh -y --module-type tcl ^python + ==> Regenerating tcl module files + +Now the ``py-scipy`` module will be: + +.. code-block:: tcl + + #%Module1.0 + ## Module file created by spack (https://github.com/LLNL/spack) on 2016-11-02 20:53:21.283547 + ## + ## py-scipy@0.18.1%gcc@6.2.0 arch=linux-Ubuntu14-x86_64-e6uljfi + ## + module-whatis "py-scipy @0.18.1" + + proc ModulesHelp { } { + puts stderr "SciPy (pronounced "Sigh Pie") is a Scientific Library for Python. It" + puts stderr "provides many user-friendly and efficient numerical routines such as" + puts stderr "routines for numerical integration and optimization." + } + + if ![ is-loaded python/2.7.12-gcc-6.2.0 ] { + puts stderr "Autoloading python/2.7.12-gcc-6.2.0" + module load python/2.7.12-gcc-6.2.0 + } + + if ![ is-loaded openblas/0.2.19-gcc-6.2.0 ] { + puts stderr "Autoloading openblas/0.2.19-gcc-6.2.0" + module load openblas/0.2.19-gcc-6.2.0 + } + + if ![ is-loaded py-numpy/1.11.1-gcc-6.2.0-openblas ] { + puts stderr "Autoloading py-numpy/1.11.1-gcc-6.2.0-openblas" + module load py-numpy/1.11.1-gcc-6.2.0-openblas + } + + prepend-path CMAKE_PREFIX_PATH "/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh/" + prepend-path LD_LIBRARY_PATH "/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh/lib" + prepend-path PYTHONPATH "/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh/lib/python2.7/site-packages" + setenv PY_SCIPY_ROOT "/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh" + conflict py-scipy + +and will contain code to autoload all the dependencies: + +.. code-block:: console + + $ module load py-scipy + Autoloading python/2.7.12-gcc-6.2.0 + Autoloading openblas/0.2.19-gcc-6.2.0 + Autoloading py-numpy/1.11.1-gcc-6.2.0-openblas + +----------------------------- +Lua hierarchical module files +----------------------------- + +In the final part of this tutorial you will modify ``modules.yaml`` to generate +Lua hierarchical module files. You will see that most of the directives used before +are also valid in the ``lmod`` context. + +^^^^^^^^^^^^^^^^^ +Core/Compiler/MPI +^^^^^^^^^^^^^^^^^ + +.. warning:: + Only LMod supports Lua hierarchical module files + For this part of the tutorial you need to be using LMod to + manage your environment. + +The most common hierarchy is the so called ``Core/Compiler/MPI``. To have an idea +how a hierarchy is organized you may refer to the +`Lmod guide <https://www.tacc.utexas.edu/research-development/tacc-projects/lmod/user-guide/module-hierarchy>`_. +Since ``lmod`` is not enabled by default, you need to add it to the list of +enabled module file generators. The other things you need to do are: + +- change the ``tcl`` tag to ``lmod`` +- remove ``tcl`` specific directives (``naming_scheme`` and ``conflict``) +- set which compilers are considered ``core`` +- remove the ``mpi`` related suffixes (as they will be substituted by hierarchies) + +After modifications the configuration file will be: + +.. code-block:: yaml + :emphasize-lines: 2-6 + + modules: + enable:: + - lmod + lmod: + core_compilers: + - 'gcc@4.8' + hash_length: 0 + whitelist: + - gcc + blacklist: + - '%gcc@4.8' + all: + suffixes: + '^openblas': openblas + '^netlib-lapack': netlib + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + environment: + set: + '${PACKAGE}_ROOT': '${PREFIX}' + gcc: + environment: + set: + CC: gcc + CXX: g++ + FC: gfortran + F90: gfortran + F77: gfortran + openmpi: + environment: + set: + SLURM_MPI_TYPE: pmi2 + OMPI_MCA_btl_openib_warn_default_gid_prefix: '0' + + +.. note:: + The double colon + The double colon after ``enable`` is intentional and it serves the + purpose of overriding the default list of enabled generators so + that only ``lmod`` will be active (see :ref:`the reference + manual <config-overrides>` for a more detailed explanation of + config scopes). + +The directive ``core_compilers`` accepts a list of compilers : everything built +using these compilers will create a module in the ``Core`` part of the hierarchy. It is +common practice to put the OS provided compilers in the list and only build common utilities +and other compilers in ``Core``. + +If you regenerate the module files + +.. code-block:: console + + $ spack module refresh --module-type lmod --delete-tree -y + +and update ``MODULEPATH`` to point to the ``Core`` folder, and +list the available modules, you'll see: + +.. code-block:: console + + $ module unuse /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 + $ module use /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core + $ module avail + + ----------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core ----------------------------------------------------------------------- + gcc/6.2.0 + +The only module visible now is ``gcc``. Loading that you will make +visible the ``Compiler`` part of the software stack that was built with ``gcc/6.2.0``: + +.. code-block:: console + + $ module load gcc + $ module avail + + -------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/gcc/6.2.0 --------------------------------------------------------------------- + binutils/2.27 curl/7.50.3 hwloc/1.11.4 libtool/2.4.6 lzo/2.09 netlib-lapack/3.6.1 openssl/1.0.2j py-scipy/0.18.1-openblas util-macros/1.19.0 + bison/3.0.4 expat/2.2.0 libarchive/3.2.1 libxml2/2.9.4 m4/1.4.17 nettle/3.2 pkg-config/0.29.1 py-setuptools/25.2.0 xz/5.2.2 + bzip2/1.0.6 flex/2.6.0 libpciaccess/0.13.4 lz4/131 mpich/3.2 openblas/0.2.19 py-nose/1.3.7 python/2.7.12 zlib/1.2.8 + cmake/3.6.1 gmp/6.1.1 libsigsegv/2.10 lzma/4.32.7 ncurses/6.0 openmpi/2.0.1 py-numpy/1.11.1-openblas sqlite/3.8.5 + + ----------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core ----------------------------------------------------------------------- + gcc/6.2.0 (L) + +The same holds true for the ``MPI`` part of the stack, that you can enable by loading +either ``mpich`` or ``openmpi``. The nice features of LMod will become evident +once you'll try switching among different stacks: + +.. code-block:: console + + $ module load mpich + $ module avail + + ----------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/mpich/3.2-5n5xoep/gcc/6.2.0 ------------------------------------------------------------ + netlib-scalapack/2.0.2-netlib netlib-scalapack/2.0.2-openblas (D) + + -------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/gcc/6.2.0 --------------------------------------------------------------------- + binutils/2.27 curl/7.50.3 hwloc/1.11.4 libtool/2.4.6 lzo/2.09 netlib-lapack/3.6.1 openssl/1.0.2j py-scipy/0.18.1-openblas util-macros/1.19.0 + bison/3.0.4 expat/2.2.0 libarchive/3.2.1 libxml2/2.9.4 m4/1.4.17 nettle/3.2 pkg-config/0.29.1 py-setuptools/25.2.0 xz/5.2.2 + bzip2/1.0.6 flex/2.6.0 libpciaccess/0.13.4 lz4/131 mpich/3.2 (L) openblas/0.2.19 py-nose/1.3.7 python/2.7.12 zlib/1.2.8 + cmake/3.6.1 gmp/6.1.1 libsigsegv/2.10 lzma/4.32.7 ncurses/6.0 openmpi/2.0.1 py-numpy/1.11.1-openblas sqlite/3.8.5 + + ----------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core ----------------------------------------------------------------------- + gcc/6.2.0 (L) + + $ module load openblas netlib-scalapack/2.0.2-openblas + $ module list + + Currently Loaded Modules: + 1) gcc/6.2.0 2) mpich/3.2 3) openblas/0.2.19 4) netlib-scalapack/2.0.2-openblas + + $ module load openmpi + + Lmod is automatically replacing "mpich/3.2" with "openmpi/2.0.1" + + + Due to MODULEPATH changes the following have been reloaded: + 1) netlib-scalapack/2.0.2-openblas + +This layout is already a great improvement over the usual non-hierarchical layout, +but it still has an asymmetry: ``LAPACK`` providers are semantically the same as ``MPI`` +providers, but they are still not part of the hierarchy. We'll see a possible solution +next. + +.. Activate lmod and turn the previous modifications into lmod: + Add core compilers + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Extend the hierarchy to other virtual providers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. warning:: + This is an experimental feature + Having a hierarchy deeper than ``Core``/``Compiler``/``MPI`` is an experimental + feature, still not fully supported by ``module spider``, + see `here <https://github.com/TACC/Lmod/issues/114>`_. Furthermore its use + with hierarchies more complex than ``Core``/``Compiler``/``MPI``/``LAPACK`` + has not been thoroughly tested in production environments. + +Spack permits you to generate Lua hierarchical module files where users +can add an arbitrary list of virtual providers to the triplet +``Core``/``Compiler``/``MPI``. A configuration file like: + +.. code-block:: yaml + :emphasize-lines: 7,8 + + modules: + enable:: + - lmod + lmod: + core_compilers: + - 'gcc@4.8' + hierarchical_scheme: + - lapack + hash_length: 0 + whitelist: + - gcc + blacklist: + - '%gcc@4.8' + - readline + all: + filter: + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] + environment: + set: + '${PACKAGE}_ROOT': '${PREFIX}' + gcc: + environment: + set: + CC: gcc + CXX: g++ + FC: gfortran + F90: gfortran + F77: gfortran + openmpi: + environment: + set: + SLURM_MPI_TYPE: pmi2 + OMPI_MCA_btl_openib_warn_default_gid_prefix: '0' + +will add ``lapack`` providers to the mix. After the usual regeneration of module files: + +.. code-block:: console + + $ module purge + $ spack module refresh --module-type lmod --delete-tree -y + ==> Regenerating lmod module files + +you will have something like: + +.. code-block:: console + + $ module load gcc + $ module load openblas + $ module load openmpi + $ module avail + + --------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/openblas/0.2.19-js33umc/openmpi/2.0.1-s3qbtby/gcc/6.2.0 ---------------------------------------------- + netlib-scalapack/2.0.2 + + -------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/openblas/0.2.19-js33umc/gcc/6.2.0 --------------------------------------------------------- + py-numpy/1.11.1 py-scipy/0.18.1 + + -------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/gcc/6.2.0 --------------------------------------------------------------------- + binutils/2.27 curl/7.50.3 hwloc/1.11.4 libtool/2.4.6 lzo/2.09 netlib-lapack/3.6.1 openssl/1.0.2j python/2.7.12 zlib/1.2.8 + bison/3.0.4 expat/2.2.0 libarchive/3.2.1 libxml2/2.9.4 m4/1.4.17 nettle/3.2 pkg-config/0.29.1 sqlite/3.8.5 + bzip2/1.0.6 flex/2.6.0 libpciaccess/0.13.4 lz4/131 mpich/3.2 openblas/0.2.19 (L) py-nose/1.3.7 util-macros/1.19.0 + cmake/3.6.1 gmp/6.1.1 libsigsegv/2.10 lzma/4.32.7 ncurses/6.0 openmpi/2.0.1 (L) py-setuptools/25.2.0 xz/5.2.2 + + ----------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core ----------------------------------------------------------------------- + gcc/6.2.0 (L) + +Now both the ``MPI`` and the ``LAPACK`` providers are handled by LMod as hierarchies: + +.. code-block:: console + + $ module load py-numpy netlib-scalapack + $ module load mpich + + Lmod is automatically replacing "openmpi/2.0.1" with "mpich/3.2" + + + Due to MODULEPATH changes the following have been reloaded: + 1) netlib-scalapack/2.0.2 + + $ module load netlib-lapack + + Lmod is automatically replacing "openblas/0.2.19" with "netlib-lapack/3.6.1" + + + Inactive Modules: + 1) py-numpy + + Due to MODULEPATH changes the following have been reloaded: + 1) netlib-scalapack/2.0.2 + +making the use of tags to differentiate them unnecessary. +Note that because we only compiled ``py-numpy`` with ``openblas`` the module +is made inactive when we switch the ``LAPACK`` provider. The user +environment will now be consistent by design! |