From 6795f4ca12369b9aee8700b65f4150f60cde460b Mon Sep 17 00:00:00 2001 From: "Adam J. Stewart" Date: Fri, 26 Aug 2016 16:41:00 -0500 Subject: Use console instead of shell syntax lexer, fixes #1634 --- lib/spack/docs/basic_usage.rst | 831 ++++++++++++++++++------------------ lib/spack/docs/configuration.rst | 62 +-- lib/spack/docs/developer_guide.rst | 104 +++-- lib/spack/docs/features.rst | 26 +- lib/spack/docs/getting_started.rst | 11 +- lib/spack/docs/index.rst | 2 +- lib/spack/docs/mirrors.rst | 104 ++--- lib/spack/docs/packaging_guide.rst | 832 +++++++++++++++++-------------------- 8 files changed, 970 insertions(+), 1002 deletions(-) diff --git a/lib/spack/docs/basic_usage.rst b/lib/spack/docs/basic_usage.rst index a3cec2cc8b..fd701789ec 100644 --- a/lib/spack/docs/basic_usage.rst +++ b/lib/spack/docs/basic_usage.rst @@ -8,14 +8,17 @@ The ``spack`` command has many *subcommands*. You'll only need a small subset of them for typical usage. Note that Spack colorizes output. ``less -R`` should be used with -Spack to maintain this colorization. E.g.:: +Spack to maintain this colorization. E.g.: - spack find | less -R +.. code-block:: console -It is recommend that the following be put in your ``.bashrc`` file:: + $ spack find | less -R - alias less='less -R' +It is recommend that the following be put in your ``.bashrc`` file: +.. code-block:: sh + + alias less='less -R' -------------------------- Listing available packages @@ -114,18 +117,18 @@ Installing and uninstalling For example, To install the latest version of the ``mpileaks`` package, you might type this: -.. code-block:: sh +.. code-block:: console $ spack install mpileaks -If `mpileaks` depends on other packages, Spack will install the +If ``mpileaks`` depends on other packages, Spack will install the dependencies first. It then fetches the ``mpileaks`` tarball, expands it, verifies that it was downloaded without errors, builds it, and installs it in its own directory under ``$SPACK_ROOT/opt``. You'll see a number of messages from spack, a lot of build output, and a message that the packages is installed: -.. code-block:: sh +.. code-block:: console $ spack install mpileaks ==> Installing mpileaks @@ -155,7 +158,7 @@ Building a specific version Spack can also build *specific versions* of a package. To do this, just add ``@`` after the package name, followed by a version: -.. code-block:: sh +.. code-block:: console $ spack install mpich@3.0.4 @@ -174,8 +177,8 @@ two configurations of the same version of a package, one built with boost This can all be done on the command line using the *spec* syntax. Spack calls the descriptor used to refer to a particular package configuration a **spec**. In the commands above, ``mpileaks`` and -``mpileaks@3.0.4``. We'll talk more about how you can use them to -customize an installation in :ref:`sec-specs`. +``mpileaks@3.0.4`` are both valid *specs*. We'll talk more about how +you can use them to customize an installation in :ref:`sec-specs`. .. _spack-uninstall: @@ -183,53 +186,56 @@ customize an installation in :ref:`sec-specs`. ``spack uninstall`` ^^^^^^^^^^^^^^^^^^^ -To uninstall a package, type ``spack uninstall ``. This will ask the user for -confirmation, and in case will completely remove the directory in which the package was installed. +To uninstall a package, type ``spack uninstall ``. This will ask +the user for confirmation, and in case will completely remove the directory +in which the package was installed. -.. code-block:: sh +.. code-block:: console - spack uninstall mpich + $ spack uninstall mpich If there are still installed packages that depend on the package to be uninstalled, spack will refuse to uninstall it. To uninstall a package and every package that depends on it, you may give the -`--dependents` option. +``--dependents`` option. -.. code-block:: sh +.. code-block:: console - spack uninstall --dependents mpich + $ spack uninstall --dependents mpich -will display a list of all the packages that depend on `mpich` and, upon confirmation, -will uninstall them in the right order. +will display a list of all the packages that depend on ``mpich`` and, upon +confirmation, will uninstall them in the right order. -A line like +A command like -.. code-block:: sh +.. code-block:: console - spack uninstall mpich + $ spack uninstall mpich -may be ambiguous, if multiple ``mpich`` configurations are installed. For example, if both -``mpich@3.0.2`` and ``mpich@3.1`` are installed, ``mpich`` could refer -to either one. Because it cannot determine which one to uninstall, -Spack will ask you either to provide a version number to remove the -ambiguity or use the ``--all`` option to uninstall all of the matching packages. +may be ambiguous if multiple ``mpich`` configurations are installed. +For example, if both ``mpich@3.0.2`` and ``mpich@3.1`` are installed, +``mpich`` could refer to either one. Because it cannot determine which +one to uninstall, Spack will ask you either to provide a version number +to remove the ambiguity or use the ``--all`` option to uninstall all of +the matching packages. -You may force uninstall a package with the `--force` option +You may force uninstall a package with the ``--force`` option -.. code-block:: sh +.. code-block:: console - spack uninstall --force mpich + $ spack uninstall --force mpich -but you risk breaking other installed packages. In general, it is safer to remove dependent -packages *before* removing their dependencies or use the `--dependents` option. +but you risk breaking other installed packages. In general, it is safer to +remove dependent packages *before* removing their dependencies or use the +``--dependents`` option. ------------------------- Seeing installed packages ------------------------- We know that ``spack list`` shows you the names of available packages, -but how do you figure out which are installed? +but how do you figure out which are already installed? .. _spack-find: @@ -244,7 +250,7 @@ of the same package with different specs. Running ``spack find`` with no arguments lists installed packages: -.. code-block:: sh +.. code-block:: console $ spack find ==> 74 installed packages. @@ -283,17 +289,18 @@ only shows the version of installed packages. ``spack find`` can filter the package list based on the package name, spec, or a number of properties of their installation status. For example, missing -dependencies of a spec can be shown with ``-m``, packages which were +dependencies of a spec can be shown with ``--missing``, packages which were explicitly installed with ``spack install `` can be singled out with -``-e`` and those which have been pulled in only as dependencies with ``-E``. +``--explicit`` and those which have been pulled in only as dependencies with +``--implicit``. In some cases, there may be different configurations of the *same* version of a package installed. For example, there are two -installations of of ``libdwarf@20130729`` above. We can look at them -in more detail using ``spack find -d``, and by asking only to show +installations of ``libdwarf@20130729`` above. We can look at them +in more detail using ``spack find --deps``, and by asking only to show ``libdwarf`` packages: -.. code-block:: sh +.. code-block:: console $ spack find --deps libdwarf ==> 2 installed packages. @@ -307,25 +314,25 @@ Now we see that the two instances of ``libdwarf`` depend on *different* versions of ``libelf``: 0.8.12 and 0.8.13. This view can become complicated for packages with many dependencies. If you just want to know whether two packages' dependencies differ, you can use -``spack find -l``: +``spack find --long``: -.. code-block:: sh +.. code-block:: console - $ spack find -l libdwarf + $ spack find --long libdwarf ==> 2 installed packages. -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- libdwarf@20130729-d9b90962 libdwarf@20130729-b52fac98 -Now the ``libwarf`` installs have hashes after their names. These are +Now the ``libdwarf`` installs have hashes after their names. These are hashes over all of the dependencies of each package. If the hashes are the same, then the packages have the same dependency configuration. If you want to know the path where each package is installed, you can -use ``spack find -p``: +use ``spack find --paths``: -.. code-block:: sh +.. code-block:: console - $ spack find -p + $ spack find --paths ==> 74 installed packages. -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- ImageMagick@6.8.9-10 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/ImageMagick@6.8.9-10-4df950dd @@ -340,9 +347,9 @@ use ``spack find -p``: And, finally, you can restrict your search to a particular package by supplying its name: -.. code-block:: sh +.. code-block:: console - $ spack find -p libelf + $ spack find --paths libelf -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- libelf@0.8.11 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/libelf@0.8.11 libelf@0.8.12 /home/gamblin2/spack/opt/linux-debian7-x86_64/gcc@4.4.7/libelf@0.8.12 @@ -353,7 +360,7 @@ by supplying its name: package. If you want to find only libelf versions greater than version 0.8.12, you could say: -.. code-block:: sh +.. code-block:: console $ spack find libelf@0.8.12: -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- @@ -362,17 +369,16 @@ package. If you want to find only libelf versions greater than version Finding just the versions of libdwarf built with a particular version of libelf would look like this: -.. code-block:: sh +.. code-block:: console - $ spack find -l libdwarf ^libelf@0.8.12 + $ spack find --long libdwarf ^libelf@0.8.12 ==> 1 installed packages. -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- libdwarf@20130729-d9b90962 We can also search for packages that have a certain attribute. For example, -``spack find -l libdwarf +debug`` will show only installations of libdwarf -with the 'debug' compile-time option enabled, while ``spack find -l +debug`` -will find every installed package with a 'debug' compile-time option enabled. +``spack find libdwarf +debug`` will show only installations of libdwarf +with the 'debug' compile-time option enabled. The full spec syntax is discussed in detail in :ref:`sec-specs`. @@ -385,7 +391,7 @@ Compiler configuration Spack has the ability to build packages with multiple compilers and compiler versions. Spack searches for compilers on your machine automatically the first time it is run. It does this by inspecting -your path. +your ``PATH``. .. _spack-compilers: @@ -394,24 +400,26 @@ your path. ^^^^^^^^^^^^^^^^^^^ You can see which compilers spack has found by running ``spack -compilers`` or ``spack compiler list``:: - - $ spack compilers - ==> Available compilers - -- gcc --------------------------------------------------------- - gcc@4.9.0 gcc@4.8.0 gcc@4.7.0 gcc@4.6.2 gcc@4.4.7 - gcc@4.8.2 gcc@4.7.1 gcc@4.6.3 gcc@4.6.1 gcc@4.1.2 - -- intel ------------------------------------------------------- - intel@15.0.0 intel@14.0.0 intel@13.0.0 intel@12.1.0 intel@10.0 - intel@14.0.3 intel@13.1.1 intel@12.1.5 intel@12.0.4 intel@9.1 - intel@14.0.2 intel@13.1.0 intel@12.1.3 intel@11.1 - intel@14.0.1 intel@13.0.1 intel@12.1.2 intel@10.1 - -- clang ------------------------------------------------------- - clang@3.4 clang@3.3 clang@3.2 clang@3.1 - -- pgi --------------------------------------------------------- - pgi@14.3-0 pgi@13.2-0 pgi@12.1-0 pgi@10.9-0 pgi@8.0-1 - pgi@13.10-0 pgi@13.1-1 pgi@11.10-0 pgi@10.2-0 pgi@7.1-3 - pgi@13.6-0 pgi@12.8-0 pgi@11.1-0 pgi@9.0-4 pgi@7.0-6 +compilers`` or ``spack compiler list``: + +.. code-block:: console + + $ spack compilers + ==> Available compilers + -- gcc --------------------------------------------------------- + gcc@4.9.0 gcc@4.8.0 gcc@4.7.0 gcc@4.6.2 gcc@4.4.7 + gcc@4.8.2 gcc@4.7.1 gcc@4.6.3 gcc@4.6.1 gcc@4.1.2 + -- intel ------------------------------------------------------- + intel@15.0.0 intel@14.0.0 intel@13.0.0 intel@12.1.0 intel@10.0 + intel@14.0.3 intel@13.1.1 intel@12.1.5 intel@12.0.4 intel@9.1 + intel@14.0.2 intel@13.1.0 intel@12.1.3 intel@11.1 + intel@14.0.1 intel@13.0.1 intel@12.1.2 intel@10.1 + -- clang ------------------------------------------------------- + clang@3.4 clang@3.3 clang@3.2 clang@3.1 + -- pgi --------------------------------------------------------- + pgi@14.3-0 pgi@13.2-0 pgi@12.1-0 pgi@10.9-0 pgi@8.0-1 + pgi@13.10-0 pgi@13.1-1 pgi@11.10-0 pgi@10.2-0 pgi@7.1-3 + pgi@13.6-0 pgi@12.8-0 pgi@11.1-0 pgi@9.0-4 pgi@7.0-6 Any of these compilers can be used to build Spack packages. More on how this is done is in :ref:`sec-specs`. @@ -432,21 +440,25 @@ An alias for ``spack compiler find``. If you do not see a compiler in this list, but you want to use it with Spack, you can simply run ``spack compiler find`` with the path to -where the compiler is installed. For example:: +where the compiler is installed. For example: + +.. code-block:: console - $ spack compiler find /usr/local/tools/ic-13.0.079 - ==> Added 1 new compiler to /Users/gamblin2/.spack/compilers.yaml - intel@13.0.079 + $ spack compiler find /usr/local/tools/ic-13.0.079 + ==> Added 1 new compiler to /Users/gamblin2/.spack/compilers.yaml + intel@13.0.079 Or you can run ``spack compiler find`` with no arguments to force auto-detection. This is useful if you do not know where compilers are installed, but you know that new compilers have been added to your -``PATH``. For example, using dotkit, you might do this:: +``PATH``. For example, using dotkit, you might do this: - $ module load gcc-4.9.0 - $ spack compiler find - ==> Added 1 new compiler to /Users/gamblin2/.spack/compilers.yaml - gcc@4.9.0 +.. code-block:: console + + $ module load gcc-4.9.0 + $ spack compiler find + ==> Added 1 new compiler to /Users/gamblin2/.spack/compilers.yaml + gcc@4.9.0 This loads the environment module for gcc-4.9.0 to add it to ``PATH``, and then it adds the compiler to Spack. @@ -458,7 +470,9 @@ This loads the environment module for gcc-4.9.0 to add it to ^^^^^^^^^^^^^^^^^^^^^^^ If you want to see specifics on a particular compiler, you can run -``spack compiler info`` on it:: +``spack compiler info`` on it: + +.. code-block:: console $ spack compiler info intel@15 intel@15.0.0: @@ -466,9 +480,11 @@ If you want to see specifics on a particular compiler, you can run cxx = /usr/local/bin/icpc-15.0.090 f77 = /usr/local/bin/ifort-15.0.090 fc = /usr/local/bin/ifort-15.0.090 + modules = [] + operating system = centos6 This shows which C, C++, and Fortran compilers were detected by Spack. -Notice also that we didn\'t have to be too specific about the +Notice also that we didn't have to be too specific about the version. We just said ``intel@15``, and information about the only matching Intel compiler was displayed. @@ -480,48 +496,54 @@ If auto-detection fails, you can manually configure a compiler by editing your ``~/.spack/compilers.yaml`` file. You can do this by running ``spack config edit compilers``, which will open the file in your ``$EDITOR``. -Each compiler configuration in the file looks like this:: +Each compiler configuration in the file looks like this: - ... - compilers: - - compiler: - modules = [] - operating_system: OS - paths: - cc: /usr/local/bin/icc-15.0.024-beta - cxx: /usr/local/bin/icpc-15.0.024-beta - f77: /usr/local/bin/ifort-15.0.024-beta - fc: /usr/local/bin/ifort-15.0.024-beta +.. code-block:: yaml - spec: intel@15.0.0: + compilers: + - compiler: + modules = [] + operating_system: centos6 + paths: + cc: /usr/local/bin/icc-15.0.024-beta + cxx: /usr/local/bin/icpc-15.0.024-beta + f77: /usr/local/bin/ifort-15.0.024-beta + fc: /usr/local/bin/ifort-15.0.024-beta + spec: intel@15.0.0: For compilers, like ``clang``, that do not support Fortran, put -``None`` for ``f77`` and ``fc``:: +``None`` for ``f77`` and ``fc``: + +.. code-block:: yaml - clang@3.3svn: - cc: /usr/bin/clang - cxx: /usr/bin/clang++ - f77: None - fc: None + paths: + cc: /usr/bin/clang + cxx: /usr/bin/clang++ + f77: None + fc: None + spec: clang@3.3svn: Once you save the file, the configured compilers will show up in the list displayed by ``spack compilers``. You can also add compiler flags to manually configured compilers. The valid flags are ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, -``ldflags``, and ``ldlibs``. For example:: - - ... - compilers: - - compiler: - ... - intel@15.0.0: - cc: /usr/local/bin/icc-15.0.024-beta - cxx: /usr/local/bin/icpc-15.0.024-beta - f77: /usr/local/bin/ifort-15.0.024-beta - fc: /usr/local/bin/ifort-15.0.024-beta - cppflags: -O3 -fPIC - ... +``ldflags``, and ``ldlibs``. For example: + +.. code-block:: yaml + + compilers: + - compiler: + modules = [] + operating_system: OS + paths: + cc: /usr/local/bin/icc-15.0.024-beta + cxx: /usr/local/bin/icpc-15.0.024-beta + f77: /usr/local/bin/ifort-15.0.024-beta + fc: /usr/local/bin/ifort-15.0.024-beta + parameters: + cppflags: -O3 -fPIC + spec: intel@15.0.0: These flags will be treated by spack as if they were enterred from the command line each time this compiler is used. The compiler wrappers @@ -541,10 +563,12 @@ Spack, that descriptor is called a *spec*. Spack uses specs to refer to a particular build configuration (or configurations) of a package. Specs are more than a package name and a version; you can use them to specify the compiler, compiler version, architecture, compile options, -and dependency options for a build. In this section, we\'ll go over +and dependency options for a build. In this section, we'll go over the full syntax of specs. -Here is an example of a much longer spec than we\'ve seen thus far:: +Here is an example of a much longer spec than we've seen thus far: + +.. code-block:: none mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt arch=bgq_os ^callpath @1.1 %gcc@4.7.2 @@ -638,7 +662,7 @@ of dependencies within a spec is that they appear *after* the root package. For example, these two specs represent exactly the same configuration: -.. code-block:: sh +.. code-block:: none mpileaks ^callpath@1.0 ^libelf@0.8.3 mpileaks ^libelf@0.8.3 ^callpath@1.0 @@ -712,7 +736,7 @@ makes available. Variants can be specified using a flexible parameter syntax ``name=``. For example, ``spack install libelf debug=True`` will install libelf build with debug flags. The names of particular variants available for a package depend on -what was provided by the package author. ``spack into `` will +what was provided by the package author. ``spack info `` will provide information on what build variants are available. For compatibility with earlier versions, variants which happen to be @@ -723,7 +747,7 @@ compile time option for the libelf package. Depending on the package a variant may have any default value. For ``libelf`` here, ``debug`` is ``False`` by default, and we turned it on -with ``debug=True`` or ``+debug``. If a package is ``True`` by default +with ``debug=True`` or ``+debug``. If a variant is ``True`` by default you can turn it off by either adding ``-name`` or ``~name`` to the spec. There are two syntaxes here because, depending on context, ``~`` and @@ -774,9 +798,9 @@ command line. From within python files, the same spec would be specified handles the quote symbols. The six compiler flags are injected in the order of implicit make commands -in gnu autotools. If all flags are set, the order is -``$cppflags $cflags|$cxxflags $ldflags command $ldlibs`` for C and C++ and -``$fflags $cppflags $ldflags command $ldlibs`` for fortran. +in GNU Autotools. If all flags are set, the order is +``$cppflags $cflags|$cxxflags $ldflags $ldlibs`` for C and C++ and +``$fflags $cppflags $ldflags $ldlibs`` for Fortran. ^^^^^^^^^^^^^^^^^^^^^^^ Architecture specifiers @@ -786,14 +810,14 @@ The architecture can be specified by using the reserved words ``target`` and/or ``os`` (``target=x86-64 os=debian7``). You can also use the triplet form of platform, operating system and processor. -.. code-block:: sh +.. code-block:: console - spack install libelf arch=cray_xc-CNL10-haswell + $ spack install libelf arch=cray-CNL10-haswell Users on non-Cray systems won't have to worry about specifying the architecture. Spack will autodetect what kind of operating system is on your machine as well as the processor. For more information on how the architecture can be -used on Cray machines, check here :ref:`cray-support` +used on Cray machines, see :ref:`cray-support` .. _sec-virtual-dependencies: @@ -848,21 +872,27 @@ Constraining virtual packages When installing a package that depends on a virtual package, you can opt to specify the particular provider you want to use, or you can let -Spack pick. For example, if you just type this:: +Spack pick. For example, if you just type this: + +.. code-block:: console - spack install mpileaks + $ spack install mpileaks Then spack will pick a provider for you according to site policies. -If you really want a particular version, say mpich, then you could -run this instead:: +If you really want a particular version, say ``mpich``, then you could +run this instead: - spack install mpileaks ^mpich +.. code-block:: console + + $ spack install mpileaks ^mpich This forces spack to use some version of ``mpich`` for its implementation. As always, you can be even more specific and require -a particular ``mpich`` version:: +a particular ``mpich`` version: + +.. code-block:: console - spack install mpileaks ^mpich@3 + $ spack install mpileaks ^mpich@3 The ``mpileaks`` package in particular only needs MPI-1 commands, so any MPI implementation will do. If another package depends on @@ -887,7 +917,7 @@ to be unique, Spack will replace the reference with the spec to which it refers. Otherwise, it will prompt for a more qualified hash. Note that this will not work to reinstall a depencency uninstalled by -``spack uninstall -f``. +``spack uninstall --force``. .. _spack-providers: @@ -937,15 +967,15 @@ distributions, this can be installed from the vendor's repository: .. code-block:: sh - yum install environment-modules # (Fedora/RHEL/CentOS) - apt-get install environment-modules # (Ubuntu/Debian) + $ yum install environment-modules # (Fedora/RHEL/CentOS) + $ apt-get install environment-modules # (Ubuntu/Debian) If your Linux distribution does not have Environment Modules, you can get it with Spack: -.. code-block:: sh +.. code-block:: console - spack install environment-modules + $ spack install environment-modules In this case to activate it automatically you need to add the following two lines to your ``.bashrc`` profile (or similar): @@ -972,7 +1002,8 @@ For ``bash`` or ``ksh``, run: .. code-block:: sh - . ${SPACK_ROOT}/share/spack/setup-env.sh + export SPACK_ROOT=/path/to/spack + . $SPACK_ROOT/share/spack/setup-env.sh For ``csh`` and ``tcsh`` run: @@ -995,8 +1026,8 @@ directories: .. code-block:: sh - ${SPACK_ROOT}/share/spack/modules - ${SPACK_ROOT}/share/spack/dotkit + $SPACK_ROOT/share/spack/modules + $SPACK_ROOT/share/spack/dotkit The directories are automatically added to your ``MODULEPATH`` and ``DK_NODE`` environment variables when you enable Spack's `shell @@ -1011,56 +1042,56 @@ If you have shell support enabled you should be able to run either been installed. Here is sample output of those programs, showing lots of installed packages. - .. code-block:: sh - - $ module avail - - ------- /home/gamblin2/spack/share/spack/modules/linux-debian7-x86_64 -------- - adept-utils@1.0%gcc@4.4.7-5adef8da libelf@0.8.13%gcc@4.4.7 - automaded@1.0%gcc@4.4.7-d9691bb0 libelf@0.8.13%intel@15.0.0 - boost@1.55.0%gcc@4.4.7 mpc@1.0.2%gcc@4.4.7-559607f5 - callpath@1.0.1%gcc@4.4.7-5dce4318 mpfr@3.1.2%gcc@4.4.7 - dyninst@8.1.2%gcc@4.4.7-b040c20e mpich@3.0.4%gcc@4.4.7 - gcc@4.9.1%gcc@4.4.7-93ab98c5 mpich@3.0.4%gcc@4.9.0 - gmp@6.0.0a%gcc@4.4.7 mrnet@4.1.0%gcc@4.4.7-72b7881d - graphlib@2.0.0%gcc@4.4.7 netgauge@2.4.6%gcc@4.9.0-27912b7b - launchmon@1.0.1%gcc@4.4.7 stat@2.1.0%gcc@4.4.7-51101207 - libNBC@1.1.1%gcc@4.9.0-27912b7b sundials@2.5.0%gcc@4.9.0-27912b7b - libdwarf@20130729%gcc@4.4.7-b52fac98 - - .. code-block:: sh - - $ use -l spack - - spack ---------- - adept-utils@1.0%gcc@4.4.7-5adef8da - adept-utils @1.0 - automaded@1.0%gcc@4.4.7-d9691bb0 - automaded @1.0 - boost@1.55.0%gcc@4.4.7 - boost @1.55.0 - callpath@1.0.1%gcc@4.4.7-5dce4318 - callpath @1.0.1 - dyninst@8.1.2%gcc@4.4.7-b040c20e - dyninst @8.1.2 - gmp@6.0.0a%gcc@4.4.7 - gmp @6.0.0a - libNBC@1.1.1%gcc@4.9.0-27912b7b - libNBC @1.1.1 - libdwarf@20130729%gcc@4.4.7-b52fac98 - libdwarf @20130729 - libelf@0.8.13%gcc@4.4.7 - libelf @0.8.13 - libelf@0.8.13%intel@15.0.0 - libelf @0.8.13 - mpc@1.0.2%gcc@4.4.7-559607f5 - mpc @1.0.2 - mpfr@3.1.2%gcc@4.4.7 - mpfr @3.1.2 - mpich@3.0.4%gcc@4.4.7 - mpich @3.0.4 - mpich@3.0.4%gcc@4.9.0 - mpich @3.0.4 - netgauge@2.4.6%gcc@4.9.0-27912b7b - netgauge @2.4.6 - sundials@2.5.0%gcc@4.9.0-27912b7b - sundials @2.5.0 +.. code-block:: console + + $ module avail + + ------- /home/gamblin2/spack/share/spack/modules/linux-debian7-x86_64 -------- + adept-utils@1.0%gcc@4.4.7-5adef8da libelf@0.8.13%gcc@4.4.7 + automaded@1.0%gcc@4.4.7-d9691bb0 libelf@0.8.13%intel@15.0.0 + boost@1.55.0%gcc@4.4.7 mpc@1.0.2%gcc@4.4.7-559607f5 + callpath@1.0.1%gcc@4.4.7-5dce4318 mpfr@3.1.2%gcc@4.4.7 + dyninst@8.1.2%gcc@4.4.7-b040c20e mpich@3.0.4%gcc@4.4.7 + gcc@4.9.1%gcc@4.4.7-93ab98c5 mpich@3.0.4%gcc@4.9.0 + gmp@6.0.0a%gcc@4.4.7 mrnet@4.1.0%gcc@4.4.7-72b7881d + graphlib@2.0.0%gcc@4.4.7 netgauge@2.4.6%gcc@4.9.0-27912b7b + launchmon@1.0.1%gcc@4.4.7 stat@2.1.0%gcc@4.4.7-51101207 + libNBC@1.1.1%gcc@4.9.0-27912b7b sundials@2.5.0%gcc@4.9.0-27912b7b + libdwarf@20130729%gcc@4.4.7-b52fac98 + +.. code-block:: console + + $ use -l spack + + spack ---------- + adept-utils@1.0%gcc@4.4.7-5adef8da - adept-utils @1.0 + automaded@1.0%gcc@4.4.7-d9691bb0 - automaded @1.0 + boost@1.55.0%gcc@4.4.7 - boost @1.55.0 + callpath@1.0.1%gcc@4.4.7-5dce4318 - callpath @1.0.1 + dyninst@8.1.2%gcc@4.4.7-b040c20e - dyninst @8.1.2 + gmp@6.0.0a%gcc@4.4.7 - gmp @6.0.0a + libNBC@1.1.1%gcc@4.9.0-27912b7b - libNBC @1.1.1 + libdwarf@20130729%gcc@4.4.7-b52fac98 - libdwarf @20130729 + libelf@0.8.13%gcc@4.4.7 - libelf @0.8.13 + libelf@0.8.13%intel@15.0.0 - libelf @0.8.13 + mpc@1.0.2%gcc@4.4.7-559607f5 - mpc @1.0.2 + mpfr@3.1.2%gcc@4.4.7 - mpfr @3.1.2 + mpich@3.0.4%gcc@4.4.7 - mpich @3.0.4 + mpich@3.0.4%gcc@4.9.0 - mpich @3.0.4 + netgauge@2.4.6%gcc@4.9.0-27912b7b - netgauge @2.4.6 + sundials@2.5.0%gcc@4.9.0-27912b7b - sundials @2.5.0 The names here should look familiar, they're the same ones from ``spack find``. You *can* use the names here directly. For example, you could type either of these commands to load the callpath module: -.. code-block:: sh +.. code-block:: console - use callpath@1.0.1%gcc@4.4.7-5dce4318 + $ use callpath@1.0.1%gcc@4.4.7-5dce4318 -.. code-block:: sh +.. code-block:: console - module load callpath@1.0.1%gcc@4.4.7-5dce4318 + $ module load callpath@1.0.1%gcc@4.4.7-5dce4318 Neither of these is particularly pretty, easy to remember, or easy to type. Luckily, Spack has its own interface for using modules @@ -1077,7 +1108,7 @@ And you can use the same shortened names you use everywhere else in Spack. For example, this will add the ``mpich`` package built with ``gcc`` to your path: -.. code-block:: sh +.. code-block:: console $ spack install mpich %gcc@4.4.7 @@ -1090,18 +1121,18 @@ Spack. For example, this will add the ``mpich`` package built with Or, similarly with modules, you could type: -.. code-block:: sh +.. code-block:: console $ spack load mpich %gcc@4.4.7 These commands will add appropriate directories to your ``PATH``, -``MANPATH``, ``CPATH``, and ``LD_LIBRARY_PATH``. When you no longer want to use -a package, you can type unload or unuse similarly: +``MANPATH``, ``CPATH``, and ``LD_LIBRARY_PATH``. When you no longer +want to use a package, you can type unload or unuse similarly: -.. code-block:: sh +.. code-block:: console - $ spack unload mpich %gcc@4.4.7 # modules - $ spack unuse mpich %gcc@4.4.7 # dotkit + $ spack unload mpich %gcc@4.4.7 # modules + $ spack unuse mpich %gcc@4.4.7 # dotkit .. note:: @@ -1116,7 +1147,7 @@ Ambiguous module names If a spec used with load/unload or use/unuse is ambiguous (i.e. more than one installed package matches it), then Spack will warn you: -.. code-block:: sh +.. code-block:: console $ spack load libelf ==> Error: Multiple matches for spec libelf. Choose one: @@ -1129,7 +1160,7 @@ identify one package. For example, above, the key differentiator is that one ``libelf`` is built with the Intel compiler, while the other used ``gcc``. You could therefore just type: -.. code-block:: sh +.. code-block:: console $ spack load libelf %intel @@ -1144,13 +1175,13 @@ and are placed in the following directories under the Spack root: .. code-block:: sh - ${SPACK_ROOT}/share/spack/modules - ${SPACK_ROOT}/share/spack/dotkit + $SPACK_ROOT/share/spack/modules + $SPACK_ROOT/share/spack/dotkit The content that gets written in each module file can be customized in two ways: - 1. overriding part of the ``spack.Package`` API within a ``package.py`` - 2. writing dedicated configuration files +#. overriding part of the ``spack.Package`` API within a ``package.py`` +#. writing dedicated configuration files ^^^^^^^^^^^^^^^^^^^^^^^^ Override ``Package`` API @@ -1161,9 +1192,9 @@ of module files: .. code-block:: python - def setup_environment(self, spack_env, run_env): - """Set up the compile and runtime environments for a package.""" - pass + def setup_environment(self, spack_env, run_env): + """Set up the compile and runtime environments for a package.""" + pass """"""""""""""""" Recursive Modules @@ -1179,9 +1210,9 @@ RPATH. Modules may be loaded recursively with the ``load`` command's ``--dependencies`` or ``-r`` argument: -.. code-block:: sh +.. code-block:: console - $ spack load --dependencies ... + $ spack load --dependencies ... More than one spec may be placed on the command line here. @@ -1195,46 +1226,46 @@ commands inside a shell script. By adding the ``--shell`` flag, ``spack module find`` may also be used to generate code that can be cut-and-pasted into a shell script. For example: -.. code-block:: sh - - $ spack module find tcl --dependencies --shell py-numpy git - # bzip2@1.0.6%gcc@4.9.3=linux-x86_64 - module load bzip2-1.0.6-gcc-4.9.3-ktnrhkrmbbtlvnagfatrarzjojmkvzsx - # ncurses@6.0%gcc@4.9.3=linux-x86_64 - module load ncurses-6.0-gcc-4.9.3-kaazyneh3bjkfnalunchyqtygoe2mncv - # zlib@1.2.8%gcc@4.9.3=linux-x86_64 - module load zlib-1.2.8-gcc-4.9.3-v3ufwaahjnviyvgjcelo36nywx2ufj7z - # sqlite@3.8.5%gcc@4.9.3=linux-x86_64 - module load sqlite-3.8.5-gcc-4.9.3-a3eediswgd5f3rmto7g3szoew5nhehbr - # readline@6.3%gcc@4.9.3=linux-x86_64 - module load readline-6.3-gcc-4.9.3-se6r3lsycrwxyhreg4lqirp6xixxejh3 - # python@3.5.1%gcc@4.9.3=linux-x86_64 - module load python-3.5.1-gcc-4.9.3-5q5rsrtjld4u6jiicuvtnx52m7tfhegi - # py-setuptools@20.5%gcc@4.9.3=linux-x86_64 - module load py-setuptools-20.5-gcc-4.9.3-4qr2suj6p6glepnedmwhl4f62x64wxw2 - # py-nose@1.3.7%gcc@4.9.3=linux-x86_64 - module load py-nose-1.3.7-gcc-4.9.3-pwhtjw2dvdvfzjwuuztkzr7b4l6zepli - # openblas@0.2.17%gcc@4.9.3+shared=linux-x86_64 - module load openblas-0.2.17-gcc-4.9.3-pw6rmlom7apfsnjtzfttyayzc7nx5e7y - # py-numpy@1.11.0%gcc@4.9.3+blas+lapack=linux-x86_64 - module load py-numpy-1.11.0-gcc-4.9.3-mulodttw5pcyjufva4htsktwty4qd52r - # curl@7.47.1%gcc@4.9.3=linux-x86_64 - module load curl-7.47.1-gcc-4.9.3-ohz3fwsepm3b462p5lnaquv7op7naqbi - # autoconf@2.69%gcc@4.9.3=linux-x86_64 - module load autoconf-2.69-gcc-4.9.3-bkibjqhgqm5e3o423ogfv2y3o6h2uoq4 - # cmake@3.5.0%gcc@4.9.3~doc+ncurses+openssl~qt=linux-x86_64 - module load cmake-3.5.0-gcc-4.9.3-x7xnsklmgwla3ubfgzppamtbqk5rwn7t - # expat@2.1.0%gcc@4.9.3=linux-x86_64 - module load expat-2.1.0-gcc-4.9.3-6pkz2ucnk2e62imwakejjvbv6egncppd - # git@2.8.0-rc2%gcc@4.9.3+curl+expat=linux-x86_64 - module load git-2.8.0-rc2-gcc-4.9.3-3bib4hqtnv5xjjoq5ugt3inblt4xrgkd +.. code-block:: console + + $ spack module find tcl --dependencies --shell py-numpy git + # bzip2@1.0.6%gcc@4.9.3=linux-x86_64 + module load bzip2-1.0.6-gcc-4.9.3-ktnrhkrmbbtlvnagfatrarzjojmkvzsx + # ncurses@6.0%gcc@4.9.3=linux-x86_64 + module load ncurses-6.0-gcc-4.9.3-kaazyneh3bjkfnalunchyqtygoe2mncv + # zlib@1.2.8%gcc@4.9.3=linux-x86_64 + module load zlib-1.2.8-gcc-4.9.3-v3ufwaahjnviyvgjcelo36nywx2ufj7z + # sqlite@3.8.5%gcc@4.9.3=linux-x86_64 + module load sqlite-3.8.5-gcc-4.9.3-a3eediswgd5f3rmto7g3szoew5nhehbr + # readline@6.3%gcc@4.9.3=linux-x86_64 + module load readline-6.3-gcc-4.9.3-se6r3lsycrwxyhreg4lqirp6xixxejh3 + # python@3.5.1%gcc@4.9.3=linux-x86_64 + module load python-3.5.1-gcc-4.9.3-5q5rsrtjld4u6jiicuvtnx52m7tfhegi + # py-setuptools@20.5%gcc@4.9.3=linux-x86_64 + module load py-setuptools-20.5-gcc-4.9.3-4qr2suj6p6glepnedmwhl4f62x64wxw2 + # py-nose@1.3.7%gcc@4.9.3=linux-x86_64 + module load py-nose-1.3.7-gcc-4.9.3-pwhtjw2dvdvfzjwuuztkzr7b4l6zepli + # openblas@0.2.17%gcc@4.9.3+shared=linux-x86_64 + module load openblas-0.2.17-gcc-4.9.3-pw6rmlom7apfsnjtzfttyayzc7nx5e7y + # py-numpy@1.11.0%gcc@4.9.3+blas+lapack=linux-x86_64 + module load py-numpy-1.11.0-gcc-4.9.3-mulodttw5pcyjufva4htsktwty4qd52r + # curl@7.47.1%gcc@4.9.3=linux-x86_64 + module load curl-7.47.1-gcc-4.9.3-ohz3fwsepm3b462p5lnaquv7op7naqbi + # autoconf@2.69%gcc@4.9.3=linux-x86_64 + module load autoconf-2.69-gcc-4.9.3-bkibjqhgqm5e3o423ogfv2y3o6h2uoq4 + # cmake@3.5.0%gcc@4.9.3~doc+ncurses+openssl~qt=linux-x86_64 + module load cmake-3.5.0-gcc-4.9.3-x7xnsklmgwla3ubfgzppamtbqk5rwn7t + # expat@2.1.0%gcc@4.9.3=linux-x86_64 + module load expat-2.1.0-gcc-4.9.3-6pkz2ucnk2e62imwakejjvbv6egncppd + # git@2.8.0-rc2%gcc@4.9.3+curl+expat=linux-x86_64 + module load git-2.8.0-rc2-gcc-4.9.3-3bib4hqtnv5xjjoq5ugt3inblt4xrgkd The script may be further edited by removing unnecessary modules. -This script may be directly executed in bash via +This script may be directly executed in bash via: -.. code-block :: sh +.. code-block:: sh - source <( spack module find tcl --dependencies --shell py-numpy git ) + source < (spack module find tcl --dependencies --shell py-numpy git) ^^^^^^^^^^^^^^^^^^^^^^^^^ Regenerating Module files @@ -1242,9 +1273,9 @@ Regenerating Module files .. code-block:: python - def setup_dependent_environment(self, spack_env, run_env, dependent_spec): - """Set up the environment of packages that depend on this one""" - pass + def setup_dependent_environment(self, spack_env, run_env, dependent_spec): + """Set up the environment of packages that depend on this one""" + pass As briefly stated in the comments, the first method lets you customize the module file content for the package you are currently writing, the second @@ -1260,10 +1291,9 @@ overrides ``setup_dependent_environment`` in the following way: .. code-block:: python - def setup_dependent_environment(self, spack_env, run_env, extension_spec): - # ... - if extension_spec.package.extends(self.spec): - run_env.prepend_path('PYTHONPATH', os.path.join(extension_spec.prefix, self.site_packages_dir)) + def setup_dependent_environment(self, spack_env, run_env, extension_spec): + if extension_spec.package.extends(self.spec): + run_env.prepend_path('PYTHONPATH', os.path.join(extension_spec.prefix, self.site_packages_dir)) to insert the appropriate ``PYTHONPATH`` modifications in the module files of python packages. @@ -1289,11 +1319,11 @@ For each module system that can be enabled a finer configuration is possible: .. code-block:: yaml - modules: - tcl: - # contains environment modules specific customizations - dotkit: - # contains dotkit specific customizations + modules: + tcl: + # contains environment modules specific customizations + dotkit: + # contains dotkit specific customizations The structure under the ``tcl`` and ``dotkit`` keys is almost equal, and will be showcased in the following by some examples. @@ -1309,24 +1339,24 @@ Considering : .. code-block:: yaml - modules: - tcl: - all: # Default addition for every package - environment: - set: - BAR: 'bar' - ^openmpi:: # A double ':' overrides previous rules - environment: - set: - BAR: 'baz' - zlib: - environment: - prepend_path: - LD_LIBRARY_PATH: 'foo' - zlib%gcc@4.8: - environment: - unset: - - FOOBAR + modules: + tcl: + all: # Default addition for every package + environment: + set: + BAR: 'bar' + ^openmpi:: # A double ':' overrides previous rules + environment: + set: + BAR: 'baz' + zlib: + environment: + prepend_path: + LD_LIBRARY_PATH: 'foo' + zlib%gcc@4.8: + environment: + unset: + - FOOBAR what will happen is that: @@ -1336,10 +1366,10 @@ what will happen is that: - any spec that satisfies ``zlib%gcc@4.8`` will additionally unset ``FOOBAR`` .. note:: - Order does matter - The modifications associated with the ``all`` keyword are always evaluated - first, no matter where they appear in the configuration file. All the other - spec constraints are instead evaluated top to bottom. + Order does matter + The modifications associated with the ``all`` keyword are always evaluated + first, no matter where they appear in the configuration file. All the other + spec constraints are instead evaluated top to bottom. """""""""""""""""""""""""""""""""""""""" Filter modifications out of module files @@ -1351,11 +1381,12 @@ modified by your dotkit modules. Then : .. code-block:: yaml - modules: - dotkit: - all: - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] # Exclude changes to any of these variables + modules: + dotkit: + all: + filter: + # Exclude changes to any of these variables + environment_blacklist: ['CPATH', 'LIBRARY_PATH'] will generate dotkit module files that will not contain modifications to either ``CPATH`` or ``LIBRARY_PATH`` and environment module files that instead will @@ -1369,20 +1400,20 @@ The following lines in ``modules.yaml``: .. code-block:: yaml - modules: - tcl: - all: - autoload: 'direct' + modules: + tcl: + all: + autoload: 'direct' will produce environment module files that will automatically load their direct dependencies. .. note:: - Allowed values for ``autoload`` statements - Allowed values for ``autoload`` statements are either ``none``, ``direct`` - or ``all``. In ``tcl`` configuration it is possible to use the option - ``prerequisites`` that accepts the same values and will add ``prereq`` - statements instead of automatically loading other modules. + Allowed values for ``autoload`` statements + Allowed values for ``autoload`` statements are either ``none``, ``direct`` + or ``all``. In ``tcl`` configuration it is possible to use the option + ``prerequisites`` that accepts the same values and will add ``prereq`` + statements instead of automatically loading other modules. """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" Blacklist or whitelist the generation of specific module files @@ -1395,10 +1426,10 @@ A configuration file like: .. code-block:: yaml - modules: - tcl: - whitelist: ['gcc', 'llvm'] # Whitelist will have precedence over blacklist - blacklist: ['%gcc@4.4.7'] # Assuming gcc@4.4.7 is the system compiler + modules: + tcl: + whitelist: ['gcc', 'llvm'] # Whitelist will have precedence over blacklist + blacklist: ['%gcc@4.4.7'] # Assuming gcc@4.4.7 is the system compiler will skip module file generation for anything that satisfies ``%gcc@4.4.7``, with the exception of specs that satisfy ``gcc`` or ``llvm``. @@ -1411,23 +1442,23 @@ A configuration file like: .. code-block:: yaml - modules: - tcl: - naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}' - all: - conflict: ['{name}', 'intel/14.0.1'] + modules: + tcl: + naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}' + all: + conflict: ['{name}', 'intel/14.0.1'] will create module files that will conflict with ``intel/14.0.1`` and with the base directory of the same module, effectively preventing the possibility to load two or more versions of the same software at the same time. .. note:: - Tokens available for the naming scheme - currently only the tokens shown in the example are available to construct - the naming scheme + Tokens available for the naming scheme + currently only the tokens shown in the example are available to construct + the naming scheme .. note:: - The ``conflict`` option is ``tcl`` specific + The ``conflict`` option is ``tcl`` specific ^^^^^^^^^^^^^^^^^^^^^^^^^ Regenerating module files @@ -1448,15 +1479,12 @@ Running ``spack module refresh`` will remove the ``share/spack/modules`` and ``share/spack/dotkit`` directories, then regenerate all module and dotkit files from scratch: -.. code-block:: sh +.. code-block:: console $ spack module refresh ==> Regenerating tcl module files. ==> Regenerating dotkit module files. - -.. _extensions: - ---------------- Filesystem Views ---------------- @@ -1496,10 +1524,10 @@ built by Spack. Views can be used for a variety of purposes including: -- A central installation in a traditional layout, eg ``/usr/local`` maintained over time by the sysadmin. -- A self-contained installation area which may for the basis of a top-level atomic versioning scheme, eg ``/opt/pro`` vs ``/opt/dev``. -- Providing an atomic and monolithic binary distribution, eg for delivery as a single tarball. -- Producing ephemeral testing or developing environments. +* A central installation in a traditional layout, eg ``/usr/local`` maintained over time by the sysadmin. +* A self-contained installation area which may for the basis of a top-level atomic versioning scheme, eg ``/opt/pro`` vs ``/opt/dev``. +* Providing an atomic and monolithic binary distribution, eg for delivery as a single tarball. +* Producing ephemeral testing or developing environments. ^^^^^^^^^^^^^^^^^^^^^^ Using Filesystem Views @@ -1514,27 +1542,27 @@ The following example creates a filesystem view based on an installed ``cmake`` package and then removes from the view the files in the ``cmake`` package while retaining its dependencies. -.. code-block:: sh +.. code-block:: console - $ spack view -v symlink myview cmake@3.5.2 - ==> Linking package: "ncurses" - ==> Linking package: "zlib" - ==> Linking package: "openssl" - ==> Linking package: "cmake" + $ spack view --verbose symlink myview cmake@3.5.2 + ==> Linking package: "ncurses" + ==> Linking package: "zlib" + ==> Linking package: "openssl" + ==> Linking package: "cmake" - $ ls myview/ - bin doc etc include lib share + $ ls myview/ + bin doc etc include lib share - $ ls myview/bin/ - captoinfo clear cpack ctest infotocap openssl tabs toe tset - ccmake cmake c_rehash infocmp ncurses6-config reset tic tput + $ ls myview/bin/ + captoinfo clear cpack ctest infotocap openssl tabs toe tset + ccmake cmake c_rehash infocmp ncurses6-config reset tic tput - $ spack view -v -d false rm myview cmake@3.5.2 - ==> Removing package: "cmake" + $ spack view --verbose --dependencies false rm myview cmake@3.5.2 + ==> Removing package: "cmake" - $ ls myview/bin/ - captoinfo c_rehash infotocap openssl tabs toe tset - clear infocmp ncurses6-config reset tic tput + $ ls myview/bin/ + captoinfo c_rehash infotocap openssl tabs toe tset + clear infocmp ncurses6-config reset tic tput ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Limitations of Filesystem Views @@ -1568,6 +1596,8 @@ particular, if two packages require the same sub-tree of dependencies, removing one package (recursively) will remove its dependencies and leave the other package broken. +.. _extensions: + --------------------------- Extensions & Python support --------------------------- @@ -1583,7 +1613,7 @@ Spack has support for this type of installation as well. In Spack, a package that can live inside the prefix of another package is called an *extension*. Suppose you have Python installed like so: -.. code-block:: sh +.. code-block:: console $ spack find python ==> 1 installed packages. @@ -1598,7 +1628,7 @@ an *extension*. Suppose you have Python installed like so: You can find extensions for your Python installation like this: -.. code-block:: sh +.. code-block:: console $ spack extensions python ==> python@2.7.8%gcc@4.4.7 arch=linux-debian7-x86_64-703c7a96 @@ -1623,11 +1653,11 @@ You can find extensions for your Python installation like this: The extensions are a subset of what's returned by ``spack list``, and they are packages like any other. They are installed into their own -prefixes, and you can see this with ``spack find -p``: +prefixes, and you can see this with ``spack find --paths``: -.. code-block:: sh +.. code-block:: console - $ spack find -p py-numpy + $ spack find --paths py-numpy ==> 1 installed packages. -- linux-debian7-x86_64 / gcc@4.4.7 -------------------------------- py-numpy@1.9.1 /g/g21/gamblin2/src/spack/opt/linux-debian7-x86_64/gcc@4.4.7/py-numpy@1.9.1-66733244 @@ -1635,7 +1665,7 @@ prefixes, and you can see this with ``spack find -p``: However, even though this package is installed, you cannot use it directly when you run ``python``: -.. code-block:: sh +.. code-block:: console $ spack load python $ python @@ -1659,14 +1689,14 @@ in your current shell. For tcl modules: -.. code-block:: sh +.. code-block:: console $ spack load python $ spack load py-numpy or, for dotkit: -.. code-block:: sh +.. code-block:: console $ spack use python $ spack use py-numpy @@ -1690,7 +1720,7 @@ installation: ``spack activate`` ^^^^^^^^^^^^^^^^^^ -.. code-block:: sh +.. code-block:: console $ spack activate py-numpy ==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=linux-debian7-x86_64-3c74eb69 for python@2.7.8%gcc@4.4.7. @@ -1707,7 +1737,7 @@ once all dependencies were activated in the ``python`` installation, If we run ``spack extensions`` again, we now see the three new packages listed as activated: -.. code-block:: sh +.. code-block:: console $ spack extensions python ==> python@2.7.8%gcc@4.4.7 arch=linux-debian7-x86_64-703c7a96 @@ -1756,7 +1786,7 @@ explicitly load their preferred version. If, for some reason, you want to activate a package *without* its dependencies, you can use ``spack activate -f``: -.. code-block:: sh +.. code-block:: console $ spack activate -f py-numpy ==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=linux-debian7-x86_64-66733244 for python@2.7.8%gcc@4.4.7. @@ -1772,18 +1802,20 @@ version of a Python module. Obviously, you may want to change that at some point. ``spack deactivate`` is the command for this. There are several variants: - * ``spack deactivate `` will deactivate a single - extension. If another activated extension depends on this one, - Spack will warn you and exit with an error. - * ``spack deactivate -f `` deactivates an extension - regardless of packages that depend on it. - * ``spack deactivate -a `` deactivates an extension and - all of its dependencies. Use ``-f`` to disregard dependents. - * ``spack deactivate -a `` deactivates *all* activated - extensions of a package. For example, to deactivate *all* python - extensions, use:: +* ``spack deactivate `` will deactivate a single + extension. If another activated extension depends on this one, + Spack will warn you and exit with an error. +* ``spack deactivate --force `` deactivates an extension + regardless of packages that depend on it. +* ``spack deactivate --all `` deactivates an extension and + all of its dependencies. Use ``--force`` to disregard dependents. +* ``spack deactivate --all `` deactivates *all* activated + extensions of a package. For example, to deactivate *all* python + extensions, use: - spack deactivate -a python + .. code-block:: console + + $ spack deactivate --all python ----------------------- Filesystem requirements @@ -1796,7 +1828,7 @@ without ``flock`` support enabled. You can determine how your filesystems are mounted with ``mount -p``. The output for a Lustre filesystem might look like this: -.. code-block:: sh +.. code-block:: console $ mount -l | grep lscratch pilsner-mds1-lnet0@o2ib100:/lsd on /p/lscratchd type lustre (rw,nosuid,noauto,_netdev,lazystatfs,flock) @@ -1808,7 +1840,7 @@ system administrator to enable ``flock``. This issue typically manifests with the error below: -.. code-block:: sh +.. code-block:: console $ ./spack find Traceback (most recent call last): @@ -1817,7 +1849,7 @@ This issue typically manifests with the error below: File "./spack", line 154,' in main return_val = command(parser, args) File "./spack/lib/spack/spack/cmd/find.py", line 170, in find - specs = set(spack.installed_db.query(**q_args)) + specs = set(spack.installed_db.query(\**q_args)) File "./spack/lib/spack/spack/database.py", line 551, in query with self.read_transaction(): File "./spack/lib/spack/spack/database.py", line 598, in __enter__ @@ -1832,12 +1864,7 @@ This issue typically manifests with the error below: A nicer error message is TBD in future versions of Spack. -<<<<<<< c36f13e44dae52086ca48e4f16b514642bc91106 - .. _cray-support: -======= -.. _spack-cray: ->>>>>>> Fix #1604 and fix #1605, title underline problems ------------- Spack on Cray @@ -1846,34 +1873,34 @@ Spack on Cray Spack differs slightly when used on a Cray system. The architecture spec can differentiate between the front-end and back-end processor and operating system. For example, on Edison at NERSC, the back-end target processor -is \"Ivy Bridge\", so you can specify to use the back-end this way: +is "Ivy Bridge", so you can specify to use the back-end this way: -.. code-block:: sh +.. code-block:: console - spack install zlib target=ivybridge + $ spack install zlib target=ivybridge You can also use the operating system to build against the back-end: -.. code-block:: sh +.. code-block:: console - spack install zlib os=CNL10 + $ spack install zlib os=CNL10 Notice that the name includes both the operating system name and the major version number concatenated together. Alternatively, if you want to build something for the front-end, you can specify the front-end target processor. The processor for a login node -on Edison is \"Sandy bridge\" so we specify on the command line like so: +on Edison is "Sandy bridge" so we specify on the command line like so: -.. code-block:: sh +.. code-block:: console - spack install zlib target=sandybridge + $ spack install zlib target=sandybridge And the front-end operating system is: -.. code-block:: sh +.. code-block:: console - spack install zlib os=SuSE11 + $ spack install zlib os=SuSE11 ^^^^^^^^^^^^^^^^^^^^^^^ Cray compiler detection @@ -1888,27 +1915,27 @@ compiler wrapper names (i.e. cc, CC, ftn). During build time, Spack will load the correct PrgEnv and compiler module and will call appropriate wrapper. The compilers.yaml config file will also differ. There is a -modules section that is filled with the compiler\'s Programming Environment -and module name. On other systems, this field is empty []:: +modules section that is filled with the compiler's Programming Environment +and module name. On other systems, this field is empty []: - ... - - compiler: - modules: - - PrgEnv-intel - - intel/15.0.109 - ... +.. code-block:: yaml + + - compiler: + modules: + - PrgEnv-intel + - intel/15.0.109 As mentioned earlier, the compiler paths will look different on a Cray system. Since most compilers are invoked using cc, CC and ftn, the paths for each -compiler are replaced with their respective Cray compiler wrapper names:: +compiler are replaced with their respective Cray compiler wrapper names: + +.. code-block:: yaml - ... paths: cc: cc cxx: CC f77: ftn fc: ftn - ... As opposed to an explicit path to the compiler executable. This allows Spack to call the Cray compiler wrappers during build time. @@ -1917,24 +1944,24 @@ For more on compiler configuration, check out :ref:`compiler-config`. Spack sets the default Cray link type to dynamic, to better match other other platforms. Individual packages can enable static linking (which is the -default outside of Spack on cray systems) using the -static flag. +default outside of Spack on cray systems) using the ``-static`` flag. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Setting defaults and using Cray modules ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If you want to use default compilers for each PrgEnv and also be able -to load cray external modules, you will need to set up a packages.yaml. +to load cray external modules, you will need to set up a ``packages.yaml``. -Here\'s an example of an external configuration for cray modules: +Here's an example of an external configuration for cray modules: .. code-block:: yaml - packages: - mpi: - modules: - mpich@7.3.1%gcc@5.2.0 arch=cray_xc-haswell-CNL10: cray-mpich - mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-haswell-CNL10: cray-mpich + packages: + mpi: + modules: + mpich@7.3.1%gcc@5.2.0 arch=cray_xc-haswell-CNL10: cray-mpich + mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-haswell-CNL10: cray-mpich This tells Spack that for whatever package that depends on mpi, load the cray-mpich module into the environment. You can then be able to use whatever @@ -1942,7 +1969,7 @@ environment variables, libraries, etc, that are brought into the environment via module load. You can set the default compiler that Spack can use for each compiler type. -If you want to use the Cray defaults, then set them under *all:* in packages.yaml. +If you want to use the Cray defaults, then set them under ``all:`` in packages.yaml. In the compiler field, set the compiler specs in your order of preference. Whenever you build with that compiler type, Spack will concretize to that version. @@ -1951,23 +1978,23 @@ Here is an example of a full packages.yaml used at NERSC .. code-block:: yaml packages: - mpi: - modules: - mpich@7.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-mpich - mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-SuSE11-ivybridge: cray-mpich - buildable: False - netcdf: - modules: - netcdf@4.3.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-netcdf - netcdf@4.3.3.1%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge: cray-netcdf - buildable: False - hdf5: - modules: - hdf5@1.8.14%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-hdf5 - hdf5@1.8.14%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge: cray-hdf5 - buildable: False - all: - compiler: [gcc@5.2.0, intel@16.0.0.109] + mpi: + modules: + mpich@7.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-mpich + mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-SuSE11-ivybridge: cray-mpich + buildable: False + netcdf: + modules: + netcdf@4.3.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-netcdf + netcdf@4.3.3.1%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge: cray-netcdf + buildable: False + hdf5: + modules: + hdf5@1.8.14%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-hdf5 + hdf5@1.8.14%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge: cray-hdf5 + buildable: False + all: + compiler: [gcc@5.2.0, intel@16.0.0.109] Here we tell spack that whenever we want to build with gcc use version 5.2.0 or if we want to build with intel compilers, use version 16.0.0.109. We add a spec @@ -1987,7 +2014,7 @@ Getting Help ^^^^^^^^^^^^^^ If you don't find what you need here, the ``help`` subcommand will -print out out a list of *all* of ``spack``'s options and subcommands: +print out out a list of *all* of spack's options and subcommands: .. command-output:: spack help @@ -1996,5 +2023,5 @@ usage information for a particular subcommand: .. command-output:: spack help install -Alternately, you can use ``spack -h`` in place of ``spack help``, or -``spack -h`` to get help on a particular subcommand. +Alternately, you can use ``spack --help`` in place of ``spack help``, or +``spack --help`` to get help on a particular subcommand. diff --git a/lib/spack/docs/configuration.rst b/lib/spack/docs/configuration.rst index 7545acd62a..0c88dd1d9d 100644 --- a/lib/spack/docs/configuration.rst +++ b/lib/spack/docs/configuration.rst @@ -10,7 +10,9 @@ Configuration Temporary space --------------- -.. warning:: Temporary space configuration will eventually be moved to +.. warning:: + + Temporary space configuration will eventually be moved to configuration files, but currently these settings are in ``lib/spack/spack/__init__.py`` @@ -73,11 +75,11 @@ directory. Here's an example of an external configuration: .. code-block:: yaml packages: - openmpi: - paths: - openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3 - openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug - openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel + openmpi: + paths: + openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3 + openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug + openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel This example lists three installations of OpenMPI, one built with gcc, one built with gcc and debug information, and another built with Intel. @@ -110,13 +112,13 @@ be: .. code-block:: yaml - packages: - openmpi: - paths: - openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3 - openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug - openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel - buildable: False + packages: + openmpi: + paths: + openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3 + openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug + openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel + buildable: False The addition of the ``buildable`` flag tells Spack that it should never build its own version of OpenMPI, and it will instead always rely on a pre-built @@ -144,18 +146,18 @@ The preferred configuration can be controlled via the Here's an example packages.yaml file that sets preferred packages: -.. code-block:: sh +.. code-block:: yaml - packages: - opencv: - compiler: [gcc@4.9] - variants: +debug - gperftools: - version: [2.2, 2.4, 2.3] - all: - compiler: [gcc@4.4.7, gcc@4.6:, intel, clang, pgi] - providers: - mpi: [mvapich, mpich, openmpi] + packages: + opencv: + compiler: [gcc@4.9] + variants: +debug + gperftools: + version: [2.2, 2.4, 2.3] + all: + compiler: [gcc@4.4.7, gcc@4.6:, intel, clang, pgi] + providers: + mpi: [mvapich, mpich, openmpi] At a high level, this example is specifying how packages should be concretized. The opencv package should prefer using gcc 4.9 and @@ -205,15 +207,15 @@ supply ``-p`` to Spack on the command line, before any subcommands. ``spack -p`` output looks like this: -.. code-block:: sh +.. code-block:: console $ spack -p graph dyninst o dyninst - |\ - | |\ - | o | libdwarf - |/ / - o | libelf + \|\ + \| \|\ + \| o \| libdwarf + \|/ / + o \| libelf / o boost diff --git a/lib/spack/docs/developer_guide.rst b/lib/spack/docs/developer_guide.rst index 5c712149b5..04ae8fe1a1 100644 --- a/lib/spack/docs/developer_guide.rst +++ b/lib/spack/docs/developer_guide.rst @@ -18,12 +18,12 @@ Overview Spack is designed with three separate roles in mind: - #. **Users**, who need to install software *without* knowing all the - details about how it is built. - #. **Packagers** who know how a particular software package is - built and encode this information in package files. - #. **Developers** who work on Spack, add new features, and try to - make the jobs of packagers and users easier. +#. **Users**, who need to install software *without* knowing all the + details about how it is built. +#. **Packagers** who know how a particular software package is + built and encode this information in package files. +#. **Developers** who work on Spack, add new features, and try to + make the jobs of packagers and users easier. Users could be end users installing software in their home directory, or administrators installing software to a shared directory on a @@ -43,9 +43,9 @@ specification. This gets us to the two key concepts in Spack's software design: - #. **Specs**: expressions for describing builds of software, and - #. **Packages**: Python modules that build software according to a - spec. +#. **Specs**: expressions for describing builds of software, and +#. **Packages**: Python modules that build software according to a + spec. A package is a template for building particular software, and a spec as a descriptor for one or more instances of that template. Users @@ -70,56 +70,55 @@ Directory Structure ------------------- So that you can familiarize yourself with the project, we'll start -with a high level view of Spack's directory structure:: +with a high level view of Spack's directory structure: - spack/ <- installation root - bin/ - spack <- main spack executable +.. code-block:: none - etc/ - spack/ <- Spack config files. - Can be overridden by files in ~/.spack. + spack/ <- installation root + bin/ + spack <- main spack executable - var/ - spack/ <- build & stage directories - repos/ <- contains package repositories - builtin/ <- pkg repository that comes with Spack - repo.yaml <- descriptor for the builtin repository - packages/ <- directories under here contain packages - cache/ <- saves resources downloaded during installs + etc/ + spack/ <- Spack config files. + Can be overridden by files in ~/.spack. - opt/ - spack/ <- packages are installed here + var/ + spack/ <- build & stage directories + repos/ <- contains package repositories + builtin/ <- pkg repository that comes with Spack + repo.yaml <- descriptor for the builtin repository + packages/ <- directories under here contain packages + cache/ <- saves resources downloaded during installs - lib/ - spack/ - docs/ <- source for this documentation - env/ <- compiler wrappers for build environment + opt/ + spack/ <- packages are installed here - external/ <- external libs included in Spack distro - llnl/ <- some general-use libraries + lib/ + spack/ + docs/ <- source for this documentation + env/ <- compiler wrappers for build environment - spack/ <- spack module; contains Python code - cmd/ <- each file in here is a spack subcommand - compilers/ <- compiler description files - test/ <- unit test modules - util/ <- common code + external/ <- external libs included in Spack distro + llnl/ <- some general-use libraries + + spack/ <- spack module; contains Python code + cmd/ <- each file in here is a spack subcommand + compilers/ <- compiler description files + test/ <- unit test modules + util/ <- common code Spack is designed so that it could live within a `standard UNIX directory hierarchy `_, so ``lib``, ``var``, and ``opt`` all contain a ``spack`` subdirectory in case Spack is installed alongside other software. Most of the interesting -parts of Spack live in ``lib/spack``. Files under ``var`` are created -as needed, so there is no ``var`` directory when you initially clone -Spack from the repository. +parts of Spack live in ``lib/spack``. Spack has *one* directory layout and there is no install process. -version and the source code. Most Python programs don't look like -this (they use distutils, ``setup.py``, etc.) but we wanted to make -Spack *very* easy to use. The simple layout spares users from the -need to install Spack into a Python environment. Many users don't -have write access to a Python installation, and installing an entire -new instance of Python to bootstrap Spack would be very complicated. +Most Python programs don't look like this (they use distutils, ``setup.py``, +etc.) but we wanted to make Spack *very* easy to use. The simple layout +spares users from the need to install Spack into a Python environment. +Many users don't have write access to a Python installation, and installing +an entire new instance of Python to bootstrap Spack would be very complicated. Users should not have to install install a big, complicated package to use the thing that's supposed to spare them from the details of big, complicated packages. The end result is that Spack works out of the @@ -283,19 +282,18 @@ Spec objects Package objects --------------- -Most spack commands -look something like this: +Most spack commands look something like this: - #. Parse an abstract spec (or specs) from the command line, - #. *Normalize* the spec based on information in package files, - #. *Concretize* the spec according to some customizable policies, - #. Instantiate a package based on the spec, and - #. Call methods (e.g., ``install()``) on the package object. +#. Parse an abstract spec (or specs) from the command line, +#. *Normalize* the spec based on information in package files, +#. *Concretize* the spec according to some customizable policies, +#. Instantiate a package based on the spec, and +#. Call methods (e.g., ``install()``) on the package object. The information in Package files is used at all stages in this process. -Conceptually, packages are overloaded. They contain +Conceptually, packages are overloaded. They contain: ------------- Stage objects diff --git a/lib/spack/docs/features.rst b/lib/spack/docs/features.rst index d746e8ad50..57629997aa 100644 --- a/lib/spack/docs/features.rst +++ b/lib/spack/docs/features.rst @@ -14,7 +14,7 @@ Simple package installation Installing the default version of a package is simple. This will install the latest version of the ``mpileaks`` package and all of its dependencies: -.. code-block:: sh +.. code-block:: console $ spack install mpileaks @@ -26,7 +26,7 @@ Spack allows installation to be customized. Users can specify the version, build compiler, compile-time options, and cross-compile platform, all on the command line. -.. code-block:: sh +.. code-block:: console # Install a particular version by appending @ $ spack install mpileaks@1.1.2 @@ -59,7 +59,7 @@ customized extensively. Suppose that ``mpileaks`` depends indirectly on ``libelf`` and ``libdwarf``. Using ``^``, users can add custom configurations for the dependencies: -.. code-block:: sh +.. code-block:: console # Install mpileaks and link it with specific versions of libelf and libdwarf $ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug ^libelf@0.8.12 ^libdwarf@20130729+debug @@ -91,7 +91,7 @@ in pure Python. For example, this command: -.. code-block:: sh +.. code-block:: console $ spack create http://www.mr511.de/software/libelf-0.8.13.tar.gz @@ -101,16 +101,26 @@ creates a simple python file: from spack import * + class Libelf(Package): - homepage = "http://www.example.com/" + """FIXME: Put a proper description of your package here.""" + + # FIXME: Add a proper url for your package's homepage here. + homepage = "http://www.example.com" url = "http://www.mr511.de/software/libelf-0.8.13.tar.gz" version('0.8.13', '4136d7b4c04df68b686570afa26988ac') - def install(self, prefix): - configure("--prefix=%s" % prefix) + # FIXME: Add dependencies if required. + # depends_on('foo') + + def install(self, spec, prefix): + # FIXME: Modify the configure line to suit your build system here. + configure('--prefix={0}'.format(prefix)) + + # FIXME: Add logic to build and install here. make() - make("install") + make('install') It doesn't take much python coding to get from there to a working package: diff --git a/lib/spack/docs/getting_started.rst b/lib/spack/docs/getting_started.rst index 47f66b429b..676697a549 100644 --- a/lib/spack/docs/getting_started.rst +++ b/lib/spack/docs/getting_started.rst @@ -9,7 +9,7 @@ Download Getting spack is easy. You can clone it from the `github repository `_ using this command: -.. code-block:: sh +.. code-block:: console $ git clone https://github.com/llnl/spack.git @@ -18,7 +18,7 @@ full path to this directory is in the ``SPACK_ROOT`` environment variable. Add ``$SPACK_ROOT/bin`` to your path and you're ready to go: -.. code-block:: sh +.. code-block:: console $ export PATH=$SPACK_ROOT/bin:$PATH $ spack install libelf @@ -26,9 +26,10 @@ go: For a richer experience, use Spack's `shell support `_: -.. code-block:: sh +.. code-block:: console # For bash users + $ export SPACK_ROOT=/path/to/spack $ . $SPACK_ROOT/share/spack/setup-env.sh # For tcsh or csh users (note you must set SPACK_ROOT) @@ -48,9 +49,9 @@ You may want to run it out of a prefix other than the git repository you cloned. The ``spack bootstrap`` command provides this functionality. To install spack in a new directory, simply type: -.. code-block:: sh +.. code-block:: console - $ spack bootstrap /my/favorite/prefix + $ spack bootstrap /my/favorite/prefix This will install a new spack script in ``/my/favorite/prefix/bin``, which you can use just like you would the regular spack script. Each diff --git a/lib/spack/docs/index.rst b/lib/spack/docs/index.rst index 603245b5dc..45efcf131f 100644 --- a/lib/spack/docs/index.rst +++ b/lib/spack/docs/index.rst @@ -28,7 +28,7 @@ Get spack from the `github repository `_ and install your first package: -.. code-block:: sh +.. code-block:: console $ git clone https://github.com/llnl/spack.git $ cd spack/bin diff --git a/lib/spack/docs/mirrors.rst b/lib/spack/docs/mirrors.rst index 41e039b838..c4d275a8fc 100644 --- a/lib/spack/docs/mirrors.rst +++ b/lib/spack/docs/mirrors.rst @@ -11,27 +11,29 @@ mirror is a URL that points to a directory, either on the local filesystem or on some server, containing tarballs for all of Spack's packages. -Here's an example of a mirror's directory structure:: - - mirror/ - cmake/ - cmake-2.8.10.2.tar.gz - dyninst/ - dyninst-8.1.1.tgz - dyninst-8.1.2.tgz - libdwarf/ - libdwarf-20130126.tar.gz - libdwarf-20130207.tar.gz - libdwarf-20130729.tar.gz - libelf/ - libelf-0.8.12.tar.gz - libelf-0.8.13.tar.gz - libunwind/ - libunwind-1.1.tar.gz - mpich/ - mpich-3.0.4.tar.gz - mvapich2/ - mvapich2-1.9.tgz +Here's an example of a mirror's directory structure: + +.. code-block:: none + + mirror/ + cmake/ + cmake-2.8.10.2.tar.gz + dyninst/ + dyninst-8.1.1.tgz + dyninst-8.1.2.tgz + libdwarf/ + libdwarf-20130126.tar.gz + libdwarf-20130207.tar.gz + libdwarf-20130729.tar.gz + libelf/ + libelf-0.8.12.tar.gz + libelf-0.8.13.tar.gz + libunwind/ + libunwind-1.1.tar.gz + mpich/ + mpich-3.0.4.tar.gz + mvapich2/ + mvapich2-1.9.tgz The structure is very simple. There is a top-level directory. The second level directories are named after packages, and the third level @@ -57,21 +59,9 @@ contains tarballs for each package, named after each package. ---------------- Mirrors are managed with the ``spack mirror`` command. The help for -``spack mirror`` looks like this:: +``spack mirror`` looks like this: - $ spack mirror -h - usage: spack mirror [-h] SUBCOMMAND ... - - positional arguments: - SUBCOMMAND - create Create a directory to be used as a spack mirror, and fill - it with package archives. - add Add a mirror to Spack. - remove Remove a mirror by name. - list Print out available mirrors to the console. - - optional arguments: - -h, --help show this help message and exit +.. command-output:: spack help mirror The ``create`` command actually builds a mirror by fetching all of its packages from the internet and checksumming them. @@ -92,7 +82,7 @@ The command will iterate through all of Spack's packages and download the safe ones into a directory structure like the one above. Here is what it looks like: -.. code-block:: bash +.. code-block:: console $ spack mirror create libelf libdwarf ==> Created new mirror in spack-mirror-2014-06-24 @@ -134,9 +124,11 @@ Normally, ``spack mirror create`` downloads all the archives it has checksums for. If you want to only create a mirror for a subset of packages, you can do that by supplying a list of package specs on the command line after ``spack mirror create``. For example, this -command:: +command: - $ spack mirror create libelf@0.8.12: boost@1.44: +.. code-block:: console + + $ spack mirror create libelf@0.8.12: boost@1.44: Will create a mirror for libelf versions greater than or equal to 0.8.12 and boost versions greater than or equal to 1.44. @@ -146,7 +138,9 @@ Mirror files ^^^^^^^^^^^^ If you have a *very* large number of packages you want to mirror, you -can supply a file with specs in it, one per line:: +can supply a file with specs in it, one per line: + +.. code-block:: console $ cat specs.txt libdwarf @@ -168,19 +162,21 @@ your site. Once you have a mirror, you need to let spack know about it. This is relatively simple. First, figure out the URL for the mirror. If it's -a file, you can use a file URL like this one:: +a file, you can use a file URL like this one: - file:///Users/gamblin2/spack-mirror-2014-06-24 +.. code-block:: none + + file:///Users/gamblin2/spack-mirror-2014-06-24 That points to the directory on the local filesystem. If it were on a web server, you could use a URL like this one: - https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24 +https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24 Spack will use the URL as the root for all of the packages it fetches. You can tell your Spack installation to use that mirror like this: -.. code-block:: bash +.. code-block:: console $ spack mirror add local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24 @@ -192,7 +188,9 @@ Each mirror has a name so that you can refer to it again later. ``spack mirror list`` --------------------- -To see all the mirrors Spack knows about, run ``spack mirror list``:: +To see all the mirrors Spack knows about, run ``spack mirror list``: + +.. code-block:: console $ spack mirror list local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24 @@ -203,7 +201,9 @@ To see all the mirrors Spack knows about, run ``spack mirror list``:: ``spack mirror remove`` ----------------------- -To remove a mirror by name:: +To remove a mirror by name, run: + +.. code-block:: console $ spack mirror remove local_filesystem $ spack mirror list @@ -213,7 +213,9 @@ To remove a mirror by name:: Mirror precedence ----------------- -Adding a mirror really adds a line in ``~/.spack/mirrors.yaml``:: +Adding a mirror really adds a line in ``~/.spack/mirrors.yaml``: + +.. code-block:: yaml mirrors: local_filesystem: file:///Users/gamblin2/spack-mirror-2014-06-24 @@ -232,12 +234,12 @@ Local Default Cache Spack caches resources that are downloaded as part of installs. The cache is a valid spack mirror: it uses the same directory structure and naming scheme as other Spack mirrors (so it can be copied anywhere and referenced with a URL -like other mirrors). The mirror is maintained locally (within the Spack -installation directory) at :file:`var/spack/cache/`. It is always enabled (and -is always searched first when attempting to retrieve files for an installation) +like other mirrors). The mirror is maintained locally (within the Spack +installation directory) at :file:`var/spack/cache/`. It is always enabled (and +is always searched first when attempting to retrieve files for an installation) but can be cleared with :ref:`purge `; the cache directory can also -be deleted manually without issue. +be deleted manually without issue. Caching includes retrieved tarball archives and source control repositories, but -only resources with an associated digest or commit ID (e.g. a revision number +only resources with an associated digest or commit ID (e.g. a revision number for SVN) will be cached. diff --git a/lib/spack/docs/packaging_guide.rst b/lib/spack/docs/packaging_guide.rst index eae928ec76..08d39a266a 100644 --- a/lib/spack/docs/packaging_guide.rst +++ b/lib/spack/docs/packaging_guide.rst @@ -12,9 +12,9 @@ have at least some familiarity with Python, and that you've read the There are two key parts of Spack: - #. **Specs**: expressions for describing builds of software, and - #. **Packages**: Python modules that describe how to build - software according to a spec. +#. **Specs**: expressions for describing builds of software, and +#. **Packages**: Python modules that describe how to build + software according to a spec. Specs allow a user to describe a *particular* build in a way that a package author can understand. Packages allow a the packager to @@ -47,7 +47,7 @@ working. Here's an example: -.. code-block:: sh +.. code-block:: console $ spack create http://www.cmake.org/files/v2.8/cmake-2.8.12.1.tar.gz @@ -62,7 +62,7 @@ strings look like for this package. Using this information, it will try to find versions, Spack prompts you to tell it how many versions you want to download and checksum: -.. code-block:: sh +.. code-block:: console $ spack create http://www.cmake.org/files/v2.8/cmake-2.8.12.1.tar.gz ==> This looks like a URL for cmake version 2.8.12.1. @@ -86,7 +86,9 @@ always choose to download just one tarball initially, and run .. note:: If ``spack create`` fails to detect the package name correctly, - you can try supplying it yourself, e.g.:: + you can try supplying it yourself, e.g.: + + .. code-block:: console $ spack create --name cmake http://www.cmake.org/files/v2.8/cmake-2.8.12.1.tar.gz @@ -104,7 +106,7 @@ always choose to download just one tarball initially, and run Let's say you download 3 tarballs: -.. code-block:: sh +.. code-block:: none Include how many checksums in the package file? (default is 5, q to abort) 3 ==> Downloading... @@ -121,27 +123,28 @@ file in your favorite ``$EDITOR``: .. code-block:: python :linenos: - # FIXME: - # This is a template package file for Spack. We've conveniently - # put "FIXME" labels next to all the things you'll want to change. # - # Once you've edited all the FIXME's, delete this whole message, - # save this file, and test out your package like this: + # This is a template package file for Spack. We've put "FIXME" + # next to all the things you'll want to change. Once you've handled + # them, you can save this file and test your package like this: # # spack install cmake # - # You can always get back here to change things with: + # You can edit this file again by typing: # # spack edit cmake # - # See the spack documentation for more information on building - # packages. + # See the Spack documentation for more information on packaging. + # If you submit this package back to Spack as a pull request, + # please first remove this boilerplate and all FIXME comments. # from spack import * + class Cmake(Package): - """FIXME: put a proper description of your package here.""" - # FIXME: add a proper url for your package's homepage here. + """FIXME: Put a proper description of your package here.""" + + # FIXME: Add a proper url for your package's homepage here. homepage = "http://www.example.com" url = "http://www.cmake.org/files/v2.8/cmake-2.8.12.1.tar.gz" @@ -165,33 +168,33 @@ done for you. In the generated package, the download ``url`` attribute is already set. All the things you still need to change are marked with -``FIXME`` labels. The first ``FIXME`` refers to the commented -instructions at the top of the file. You can delete these -instructions after reading them. The rest of them are as follows: +``FIXME`` labels. You can delete the commented instructions between +the license and the first import statement after reading them. +The rest of the tasks you need to do are as follows: - #. Add a description. +#. Add a description. - Immediately inside the package class is a *docstring* in - triple-quotes (``"""``). It's used to generate the description - shown when users run ``spack info``. + Immediately inside the package class is a *docstring* in + triple-quotes (``"""``). It's used to generate the description + shown when users run ``spack info``. - #. Change the ``homepage`` to a useful URL. +#. Change the ``homepage`` to a useful URL. - The ``homepage`` is displayed when users run ``spack info`` so - that they can learn about packages. + The ``homepage`` is displayed when users run ``spack info`` so + that they can learn about packages. - #. Add ``depends_on()`` calls for the package's dependencies. +#. Add ``depends_on()`` calls for the package's dependencies. - ``depends_on`` tells Spack that other packages need to be built - and installed before this one. See `dependencies_`. + ``depends_on`` tells Spack that other packages need to be built + and installed before this one. See :ref:`dependencies`. - #. Get the ``install()`` method working. +#. Get the ``install()`` method working. - The ``install()`` method implements the logic to build a - package. The code should look familiar; it is designed to look - like a shell script. Specifics will differ depending on the package, - and :ref:`implementing the install method ` is - covered in detail later. + The ``install()`` method implements the logic to build a + package. The code should look familiar; it is designed to look + like a shell script. Specifics will differ depending on the package, + and :ref:`implementing the install method ` is + covered in detail later. Before going into details, we'll cover a few more basics. @@ -205,41 +208,41 @@ One of the easiest ways to learn to write packages is to look at existing ones. You can edit a package file by name with the ``spack edit`` command: -.. code-block:: sh +.. code-block:: console - spack edit cmake + $ spack edit cmake So, if you used ``spack create`` to create a package, then saved and closed the resulting file, you can get back to it with ``spack edit``. The ``cmake`` package actually lives in -``$SPACK_ROOT/var/spack/repos/builtin/packages/cmake/package.py``, but this provides -a much simpler shortcut and saves you the trouble of typing the full -path. +``$SPACK_ROOT/var/spack/repos/builtin/packages/cmake/package.py``, +but this provides a much simpler shortcut and saves you the trouble +of typing the full path. If you try to edit a package that doesn't exist, Spack will recommend -using ``spack create`` or ``spack edit -f``: +using ``spack create`` or ``spack edit --force``: -.. code-block:: sh +.. code-block:: console $ spack edit foo ==> Error: No package 'foo'. Use spack create, or supply -f/--force to edit a new file. .. _spack-edit-f: -^^^^^^^^^^^^^^^^^ -``spack edit -f`` -^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^ +``spack edit --force`` +^^^^^^^^^^^^^^^^^^^^^^ -``spack edit -f`` can be used to create a new, minimal boilerplate +``spack edit --force`` can be used to create a new, minimal boilerplate package: -.. code-block:: sh +.. code-block:: console $ spack edit -f foo Unlike ``spack create``, which infers names and versions, and which actually downloads the tarball and checksums it for you, ``spack edit --f`` has no such fanciness. It will substitute dummy values for you +--force`` has no such fanciness. It will substitute dummy values for you to fill in yourself: .. code-block:: python @@ -256,7 +259,7 @@ to fill in yourself: version('1.0', '0123456789abcdef0123456789abcdef') def install(self, spec, prefix): - configure("--prefix=" + prefix) + configure("--prefix=%s" % prefix) make() make("install") @@ -267,16 +270,9 @@ version of your package from the archive URL. Naming & directory structure ---------------------------- -.. note:: - - Spack's default naming and directory structure will change in - version 0.9. Specifically, 0.9 will stop using directory names - with special characters like ``@``, to avoid interfering with - certain packages that do not handle this well. - This section describes how packages need to be named, and where they -live in Spack's directory structure. In general, `spack-create`_ and -`spack-edit`_ handle creating package files for you, so you can skip +live in Spack's directory structure. In general, :ref:`spack-create` and +:ref:`spack-edit` handle creating package files for you, so you can skip most of the details here. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -291,13 +287,15 @@ Packages themselves live in ``$SPACK_ROOT/var/spack/repos/builtin/packages``. If you ``cd`` to that directory, you will see directories for each package: -.. command-output:: cd $SPACK_ROOT/var/spack/repos/builtin/packages; ls -CF +.. command-output:: cd $SPACK_ROOT/var/spack/repos/builtin/packages && ls :shell: :ellipsis: 10 Each directory contains a file called ``package.py``, which is where all the python code for the package goes. For example, the ``libelf`` -package lives in:: +package lives in: + +.. code-block:: none $SPACK_ROOT/var/spack/repos/builtin/packages/libelf/package.py @@ -312,7 +310,7 @@ Packages are named after the directory containing ``package.py``. It is preferred, but not required, that the directory, and thus the package name, are lower case. So, ``libelf``'s ``package.py`` lives in a directory called ``libelf``. The ``package.py`` file defines a class called ``Libelf``, which -extends Spack's ``Package`` class. for example, here is +extends Spack's ``Package`` class. For example, here is ``$SPACK_ROOT/var/spack/repos/builtin/packages/libelf/package.py``: .. code-block:: python @@ -334,7 +332,7 @@ The **directory name** (``libelf``) determines the package name that users should provide on the command line. e.g., if you type any of these: -.. code-block:: sh +.. code-block:: console $ spack install libelf $ spack install libelf@0.8.13 @@ -373,7 +371,7 @@ some examples: ================= ================= In general, you won't have to remember this naming convention because -`spack-create`_ and `spack-edit`_ handle the details for you. +:ref:`spack-create` and :ref:`spack-edit` handle the details for you. ------------------- Adding new versions @@ -482,7 +480,7 @@ to use based on the hash length. If you have one or more files to checksum, you can use the ``spack md5`` command to do it: -.. code-block:: sh +.. code-block:: console $ spack md5 foo-8.2.1.tar.gz foo-8.2.2.tar.gz ==> 2 MD5 checksums: @@ -492,7 +490,7 @@ command to do it: ``spack md5`` also accepts one or more URLs and automatically downloads the files for you: -.. code-block:: sh +.. code-block:: console $ spack md5 http://example.com/foo-8.2.1.tar.gz ==> Trying to fetch from http://example.com/foo-8.2.1.tar.gz @@ -514,7 +512,7 @@ If you want to add new versions to a package you've already created, this is automated with the ``spack checksum`` command. Here's an example for ``libelf``: -.. code-block:: sh +.. code-block:: console $ spack checksum libelf ==> Found 16 versions of libelf. @@ -538,7 +536,7 @@ they're released). It fetches the tarballs you ask for and prints out a list of ``version`` commands ready to copy/paste into your package file: -.. code-block:: sh +.. code-block:: console ==> Checksummed new versions of libelf: version('0.8.13', '4136d7b4c04df68b686570afa26988ac') @@ -605,7 +603,7 @@ same thing every time you fetch a particular version. Life isn't always simple, though, so this is not strictly enforced. When fetching from from the branch corresponding to the development version -(often called ``master``,``trunk`` or ``dev``), it is recommended to +(often called ``master``, ``trunk``, or ``dev``), it is recommended to call this version ``develop``. Spack has special treatment for this version so that ``@develop`` will satisfy dependencies like ``depends_on(abc, when="@x.y.z:")``. In other words, ``@develop`` is @@ -627,11 +625,11 @@ Git Git fetching is enabled with the following parameters to ``version``: - * ``git``: URL of the git repository. - * ``tag``: name of a tag to fetch. - * ``branch``: name of a branch to fetch. - * ``commit``: SHA hash (or prefix) of a commit to fetch. - * ``submodules``: Also fetch submodules when checking out this repository. +* ``git``: URL of the git repository. +* ``tag``: name of a tag to fetch. +* ``branch``: name of a branch to fetch. +* ``commit``: SHA hash (or prefix) of a commit to fetch. +* ``submodules``: Also fetch submodules when checking out this repository. Only one of ``tag``, ``branch``, or ``commit`` can be used at a time. @@ -675,7 +673,7 @@ Commits version('2014-10-08', git='https://github.com/example-project/example.git', commit='9d38cd4e2c94c3cea97d0e2924814acc') - This doesn't have to be a full hash; You can abbreviate it as you'd + This doesn't have to be a full hash; you can abbreviate it as you'd expect with git: .. code-block:: python @@ -705,14 +703,13 @@ Installing You can fetch and install any of the versions above as you'd expect, by using ``@`` in a spec: -.. code-block:: sh +.. code-block:: console - spack install example@2014-10-08 + $ spack install example@2014-10-08 Git and other VCS versions will show up in the list of versions when a user runs ``spack info ``. - .. _hg-fetch: ^^^^^^^^^ @@ -858,12 +855,12 @@ For example, let's take a look at the package for the PGI compilers. .. code-block:: python - # Licensing - license_required = True - license_comment = '#' - license_files = ['license.dat'] - license_vars = ['PGROUPD_LICENSE_FILE', 'LM_LICENSE_FILE'] - license_url = 'http://www.pgroup.com/doc/pgiinstall.pdf' + # Licensing + license_required = True + license_comment = '#' + license_files = ['license.dat'] + license_vars = ['PGROUPD_LICENSE_FILE', 'LM_LICENSE_FILE'] + license_url = 'http://www.pgroup.com/doc/pgiinstall.pdf' As you can see, PGI requires a license. Its license manager, FlexNet, uses the ``#`` symbol to denote a comment. It expects the license file to be @@ -886,39 +883,39 @@ this: .. code-block:: sh - # A license is required to use pgi. - # - # The recommended solution is to store your license key in this global - # license file. After installation, the following symlink(s) will be - # added to point to this file (relative to the installation prefix): - # - # license.dat - # - # Alternatively, use one of the following environment variable(s): - # - # PGROUPD_LICENSE_FILE - # LM_LICENSE_FILE - # - # If you choose to store your license in a non-standard location, you may - # set one of these variable(s) to the full pathname to the license file, or - # port@host if you store your license keys on a dedicated license server. - # You will likely want to set this variable in a module file so that it - # gets loaded every time someone tries to use pgi. - # - # For further information on how to acquire a license, please refer to: - # - # http://www.pgroup.com/doc/pgiinstall.pdf - # - # You may enter your license below. + # A license is required to use pgi. + # + # The recommended solution is to store your license key in this global + # license file. After installation, the following symlink(s) will be + # added to point to this file (relative to the installation prefix): + # + # license.dat + # + # Alternatively, use one of the following environment variable(s): + # + # PGROUPD_LICENSE_FILE + # LM_LICENSE_FILE + # + # If you choose to store your license in a non-standard location, you may + # set one of these variable(s) to the full pathname to the license file, or + # port@host if you store your license keys on a dedicated license server. + # You will likely want to set this variable in a module file so that it + # gets loaded every time someone tries to use pgi. + # + # For further information on how to acquire a license, please refer to: + # + # http://www.pgroup.com/doc/pgiinstall.pdf + # + # You may enter your license below. You can add your license directly to this file, or tell FlexNet to use a license stored on a separate license server. Here is an example that points to a license server called licman1: -.. code-block:: sh +.. code-block:: none - SERVER licman1.mcs.anl.gov 00163eb7fba5 27200 - USE_SERVER + SERVER licman1.mcs.anl.gov 00163eb7fba5 27200 + USE_SERVER If your package requires the license to install, you can reference the location of this global license using ``self.global_license_file``. @@ -949,7 +946,7 @@ after it's downloaded. ^^^^^^^^^ You can specify patches in your package file with the ``patch()`` -function. ``patch`` looks like this: +directive. ``patch`` looks like this: .. code-block:: python @@ -961,7 +958,9 @@ The first argument can be either a URL or a filename. It specifies a patch file that should be applied to your source. If the patch you supply is a filename, then the patch needs to live within the spack source tree. For example, the patch above lives in a directory -structure like this:: +structure like this: + +.. code-block:: none $SPACK_ROOT/var/spack/repos/builtin/packages/ mvapich2/ @@ -982,52 +981,58 @@ from the URL and then applied to your source code. ``patch`` can take two options keyword arguments. They are: +"""""""" ``when`` - If supplied, this is a spec that tells spack when to apply - the patch. If the installed package spec matches this spec, the - patch will be applied. In our example above, the patch is applied - when mvapich is at version ``1.9`` or higher. +"""""""" + +If supplied, this is a spec that tells spack when to apply +the patch. If the installed package spec matches this spec, the +patch will be applied. In our example above, the patch is applied +when mvapich is at version ``1.9`` or higher. +""""""""" ``level`` - This tells spack how to run the ``patch`` command. By default, - the level is 1 and spack runs ``patch -p1``. If level is 2, - spack will run ``patch -p2``, and so on. - - A lot of people are confused by level, so here's a primer. If you - look in your patch file, you may see something like this: - - .. code-block:: diff - :linenos: - - --- a/src/mpi/romio/adio/ad_lustre/ad_lustre_rwcontig.c 2013-12-10 12:05:44.806417000 -0800 - +++ b/src/mpi/romio/adio/ad_lustre/ad_lustre_rwcontig.c 2013-12-10 11:53:03.295622000 -0800 - @@ -8,7 +8,7 @@ - * Copyright (C) 2008 Sun Microsystems, Lustre group - */ - - -#define _XOPEN_SOURCE 600 - +//#define _XOPEN_SOURCE 600 - #include - #include - #include "ad_lustre.h" - - Lines 1-2 show paths with synthetic ``a/`` and ``b/`` prefixes. These - are placeholders for the two ``mvapich2`` source directories that - ``diff`` compared when it created the patch file. This is git's - default behavior when creating patch files, but other programs may - behave differently. - - ``-p1`` strips off the first level of the prefix in both paths, - allowing the patch to be applied from the root of an expanded mvapich2 - archive. If you set level to ``2``, it would strip off ``src``, and - so on. - - It's generally easier to just structure your patch file so that it - applies cleanly with ``-p1``, but if you're using a patch you didn't - create yourself, ``level`` can be handy. +""""""""" + +This tells spack how to run the ``patch`` command. By default, +the level is 1 and spack runs ``patch -p 1``. If level is 2, +spack will run ``patch -p 2``, and so on. + +A lot of people are confused by level, so here's a primer. If you +look in your patch file, you may see something like this: + +.. code-block:: diff + :linenos: + + --- a/src/mpi/romio/adio/ad_lustre/ad_lustre_rwcontig.c 2013-12-10 12:05:44.806417000 -0800 + +++ b/src/mpi/romio/adio/ad_lustre/ad_lustre_rwcontig.c 2013-12-10 11:53:03.295622000 -0800 + @@ -8,7 +8,7 @@ + * Copyright (C) 2008 Sun Microsystems, Lustre group + \*/ + + -#define _XOPEN_SOURCE 600 + +//#define _XOPEN_SOURCE 600 + #include + #include + #include "ad_lustre.h" + +Lines 1-2 show paths with synthetic ``a/`` and ``b/`` prefixes. These +are placeholders for the two ``mvapich2`` source directories that +``diff`` compared when it created the patch file. This is git's +default behavior when creating patch files, but other programs may +behave differently. + +``-p1`` strips off the first level of the prefix in both paths, +allowing the patch to be applied from the root of an expanded mvapich2 +archive. If you set level to ``2``, it would strip off ``src``, and +so on. + +It's generally easier to just structure your patch file so that it +applies cleanly with ``-p1``, but if you're using a patch you didn't +create yourself, ``level`` can be handy. ^^^^^^^^^^^^^^^^^^^^^ -``patch()`` functions +Patch functions ^^^^^^^^^^^^^^^^^^^^^ In addition to supplying patch files, you can write a custom function @@ -1037,35 +1042,10 @@ handles ``RPATH``: .. _pyside-patch: -.. code-block:: python +.. literalinclude:: ../../../var/spack/repos/builtin/packages/py-pyside/package.py + :pyobject: PyPyside.patch :linenos: - class PyPyside(Package): - ... - - def patch(self): - """Undo PySide RPATH handling and add Spack RPATH.""" - # Figure out the special RPATH - pypkg = self.spec['python'].package - rpath = self.rpath - rpath.append(os.path.join(self.prefix, pypkg.site_packages_dir, 'PySide')) - - # Add Spack's standard CMake args to the sub-builds. - # They're called BY setup.py so we have to patch it. - filter_file( - r'OPTION_CMAKE,', - r'OPTION_CMAKE, ' + ( - '"-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=FALSE", ' - '"-DCMAKE_INSTALL_RPATH=%s",' % ':'.join(rpath)), - 'setup.py') - - # PySide tries to patch ELF files to remove RPATHs - # Disable this and go with the one we set. - filter_file( - r'^\s*rpath_cmd\(pyside_path, srcpath\)', - r'#rpath_cmd(pyside_path, srcpath)', - 'pyside_postinstall.py') - A ``patch`` function, if present, will be run after patch files are applied and before ``install()`` is run. @@ -1085,9 +1065,9 @@ dependencies are found when it runs. It does this using `RPATHs `_. An RPATH is a search path, stored in a binary (an executable or library), that tells the dynamic loader where to find its dependencies at runtime. You may be -familiar with ```LD_LIBRARY_PATH`` +familiar with `LD_LIBRARY_PATH `_ -on Linux or ```DYLD_LIBRARY_PATH`` +on Linux or `DYLD_LIBRARY_PATH ` on Mac OS X. RPATH is similar to these paths, in that it tells the loader where to find libraries. Unlike them, it is embedded in @@ -1095,32 +1075,32 @@ the binary and not set in each user's environment. RPATHs in Spack are handled in one of three ways: - 1. For most packages, RPATHs are handled automatically using Spack's - :ref:`compiler wrappers `. These wrappers are - set in standard variables like ``CC``, ``CXX``, and ``FC``, so - most build systems (autotools and many gmake systems) pick them - up and use them. - 2. CMake also respects Spack's compiler wrappers, but many CMake - builds have logic to overwrite RPATHs when binaries are - installed. Spack provides the ``std_cmake_args`` variable, which - includes parameters necessary for CMake build use the right - installation RPATH. It can be used like this when ``cmake`` is - invoked: - - .. code-block:: python - - class MyPackage(Package): - ... - def install(self, spec, prefix): - cmake('..', *std_cmake_args) - make() - make('install') - - 3. If you need to modify the build to add your own RPATHs, you can - use the ``self.rpath`` property of your package, which will - return a list of all the RPATHs that Spack will use when it - links. You can see this how this is used in the :ref:`PySide - example ` above. +#. For most packages, RPATHs are handled automatically using Spack's + :ref:`compiler wrappers `. These wrappers are + set in standard variables like ``CC``, ``CXX``, ``F77``, and ``FC``, + so most build systems (autotools and many gmake systems) pick them + up and use them. +#. CMake also respects Spack's compiler wrappers, but many CMake + builds have logic to overwrite RPATHs when binaries are + installed. Spack provides the ``std_cmake_args`` variable, which + includes parameters necessary for CMake build use the right + installation RPATH. It can be used like this when ``cmake`` is + invoked: + + .. code-block:: python + + class MyPackage(Package): + ... + def install(self, spec, prefix): + cmake('..', *std_cmake_args) + make() + make('install') + +#. If you need to modify the build to add your own RPATHs, you can + use the ``self.rpath`` property of your package, which will + return a list of all the RPATHs that Spack will use when it + links. You can see this how this is used in the :ref:`PySide + example ` above. -------------------- Finding new versions @@ -1133,6 +1113,7 @@ You've already seen the ``homepage`` and ``url`` package attributes: from spack import * + class Mpich(Package): """MPICH is a high performance and widely portable implementation of the Message Passing Interface (MPI) standard.""" @@ -1194,7 +1175,9 @@ the ``list_url``, because that is where links to old versions are: versions on a single webpage, but not all do. For example, ``mpich`` has a tarball URL that looks like this: - url = "http://www.mpich.org/static/downloads/3.0.4/mpich-3.0.4.tar.gz" +.. code-block:: python + + url = "http://www.mpich.org/static/downloads/3.0.4/mpich-3.0.4.tar.gz" But its downloads are in many different subdirectories of ``http://www.mpich.org/static/downloads/``. So, we need to add a @@ -1344,7 +1327,7 @@ Or a requirement for a particular variant or compiler flags: depends_on("libelf@0.8+debug") depends_on('libelf debug=True') - depends_on('libelf cppflags="-fPIC") + depends_on('libelf cppflags="-fPIC"') Both users *and* package authors can use the same spec syntax to refer to different package configurations. Users use the spec syntax on the @@ -1356,9 +1339,9 @@ Additionally, dependencies may be specified for specific use cases: .. code-block:: python - depends_on("cmake", type="build") - depends_on("libelf", type=("build", "link")) - depends_on("python", type="run") + depends_on("cmake", type="build") + depends_on("libelf", type=("build", "link")) + depends_on("python", type="run") The dependency types are: @@ -1405,34 +1388,16 @@ packages that depend on a particular Qt installation will find it. The arguments to this function are: - * **module**: the module of the dependent package, where global - properties can be assigned. - * **spec**: the spec of the *dependency package* (the one the function is called on). - * **dep_spec**: the spec of the dependent package (i.e. dep_spec depends on spec). +* **module**: the module of the dependent package, where global + properties can be assigned. +* **spec**: the spec of the *dependency package* (the one the function is called on). +* **dep_spec**: the spec of the dependent package (i.e. dep_spec depends on spec). A good example of using these is in the Python package: -.. code-block:: python - - def setup_dependent_environment(self, module, spec, dep_spec): - # Python extension builds can have a global python executable function - module.python = Executable(join_path(spec.prefix.bin, 'python')) - - # Add variables for lib/pythonX.Y and lib/pythonX.Y/site-packages dirs. - module.python_lib_dir = os.path.join(dep_spec.prefix, self.python_lib_dir) - module.python_include_dir = os.path.join(dep_spec.prefix, self.python_include_dir) - module.site_packages_dir = os.path.join(dep_spec.prefix, self.site_packages_dir) - - # Make the site packages directory if it does not exist already. - mkdirp(module.site_packages_dir) - - # Set PYTHONPATH to include site-packages dir for the - # extension and any other python extensions it depends on. - python_paths = [] - for d in dep_spec.traverse(): - if d.package.extends(self.spec): - python_paths.append(os.path.join(d.prefix, self.site_packages_dir)) - os.environ['PYTHONPATH'] = ':'.join(python_paths) +.. literalinclude:: ../../../var/spack/repos/builtin/packages/python/package.py + :pyobject: Python.setup_dependent_environment + :linenos: The first thing that happens here is that the ``python`` command is inserted into module scope of the dependent. This allows most python @@ -1441,7 +1406,7 @@ packages to have a very simple install method, like this: .. code-block:: python def install(self, spec, prefix): - python('setup.py', 'install', '--prefix=%s' % prefix) + python('setup.py', 'install', '--prefix={0}'.format(prefix)) Python's ``setup_dependent_environment`` method also sets up some other variables, creates a directory, and sets up the ``PYTHONPATH`` @@ -1483,7 +1448,7 @@ activate``. When it is activated, all the files in its prefix will be symbolically linked into the prefix of the python package. Many packages produce Python extensions for *some* variants, but not -others: they should extend ``python`` only if the apropriate +others: they should extend ``python`` only if the appropriate variant(s) are selected. This may be accomplished with conditional ``extends()`` declarations: @@ -1502,13 +1467,15 @@ when it does the activation: .. code-block:: python - class PyNose(Package): + class PySncosmo(Package): ... - extends('python', ignore=r'bin/nosetests.*$') + # py-sncosmo binaries are duplicates of those from py-astropy + extends('python', ignore=r'bin/.*') + depends_on('py-astropy') ... -The code above will prevent ``$prefix/bin/nosetests`` from being -linked in at activation time. +The code above will prevent everything in the ``$prefix/bin/`` directory +from being linked in at activation time. .. note:: @@ -1537,15 +1504,9 @@ same way that Python does. Let's look at Python's activate function: -.. code-block:: python - - def activate(self, ext_pkg, **kwargs): - kwargs.update(ignore=self.python_ignore(ext_pkg, kwargs)) - super(Python, self).activate(ext_pkg, **kwargs) - - exts = spack.install_layout.extension_map(self.spec) - exts[ext_pkg.name] = ext_pkg.spec - self.write_easy_install_pth(exts) +.. literalinclude:: ../../../var/spack/repos/builtin/packages/python/package.py + :pyobject: Python.activate + :linenos: This function is called on the *extendee* (Python). It first calls ``activate`` in the superclass, which handles symlinking the @@ -1555,16 +1516,9 @@ Python's setuptools. Deactivate behaves similarly to activate, but it unlinks files: -.. code-block:: python - - def deactivate(self, ext_pkg, **kwargs): - kwargs.update(ignore=self.python_ignore(ext_pkg, kwargs)) - super(Python, self).deactivate(ext_pkg, **kwargs) - - exts = spack.install_layout.extension_map(self.spec) - if ext_pkg.name in exts: # Make deactivate idempotent. - del exts[ext_pkg.name] - self.write_easy_install_pth(exts) +.. literalinclude:: ../../../var/spack/repos/builtin/packages/python/package.py + :pyobject: Python.deactivate + :linenos: Both of these methods call some custom functions in the Python package. See the source for Spack's Python package for details. @@ -1708,7 +1662,7 @@ the package ``foo`` declares this: Suppose a user invokes ``spack install`` like this: -.. code-block:: sh +.. code-block:: console $ spack install foo ^mpich@1.0 @@ -1726,9 +1680,9 @@ command line ` and within package definitions, we can talk about how Spack puts all of this information together. When you run this: -.. code-block:: sh +.. code-block:: console - spack install mpileaks ^callpath@1.0+debug ^libelf@0.8.11 + $ spack install mpileaks ^callpath@1.0+debug ^libelf@0.8.11 Spack parses the command line and builds a spec from the description. The spec says that ``mpileaks`` should be built with the ``callpath`` @@ -1744,7 +1698,9 @@ abstract spec is partially specified. In other words, it could describe more than one build of a package. Spack does this to make things easier on the user: they should only have to specify as much of the package spec as they care about. Here's an example partial spec -DAG, based on the constraints above:: +DAG, based on the constraints above: + +.. code-block:: none mpileaks ^callpath@1.0+debug @@ -1780,7 +1736,9 @@ remaining unspecified parts in order to install. This process is called **concretization**. Concretization happens in between the time the user runs ``spack install`` and the time the ``install()`` method is called. The concretized version of the spec above might look like -this:: +this: + +.. code-block:: none mpileaks@2.3%gcc@4.7.3 arch=linux-debian7-x86_64 ^callpath@1.0%gcc@4.7.3+debug arch=linux-debian7-x86_64 @@ -1819,7 +1777,7 @@ the preferences of their own users. For an arbitrary spec, you can see the result of concretization by running ``spack spec``. For example: -.. code-block:: sh +.. code-block:: console $ spack spec dyninst@8.0.1 dyninst@8.0.1 @@ -1857,7 +1815,7 @@ where the real work of installation happens, and it's the main part of the package you'll need to customize for each piece of software. .. literalinclude:: ../../../var/spack/repos/builtin/packages/libelf/package.py - :start-after: 0.8.12 + :pyobject: Libelf.install :linenos: ``install`` takes a ``spec``: a description of how the package should @@ -1945,8 +1903,8 @@ Environment variables Spack sets a number of standard environment variables that serve two purposes: - #. Make build systems use Spack's compiler wrappers for their builds. - #. Allow build systems to find dependencies more easily +#. Make build systems use Spack's compiler wrappers for their builds. +#. Allow build systems to find dependencies more easily The Compiler environment variables that Spack sets are: @@ -1960,7 +1918,7 @@ The Compiler environment variables that Spack sets are: ============ =============================== All of these are standard variables respected by most build systems. -If your project uses ``autotools`` or ``CMake``, then it should pick +If your project uses ``Autotools`` or ``CMake``, then it should pick them up automatically when you run ``configure`` or ``cmake`` in the ``install()`` function. Many traditional builds using GNU Make and BSD make also respect these variables, so they may work with these @@ -1977,12 +1935,12 @@ In addition to the compiler variables, these variables are set before entering ``install()`` so that packages can locate dependencies easily: - ======================= ============================= - ``PATH`` Set to point to ``/bin`` directories of dependencies - ``CMAKE_PREFIX_PATH`` Path to dependency prefixes for CMake - ``PKG_CONFIG_PATH`` Path to any pkgconfig directories for dependencies - ``PYTHONPATH`` Path to site-packages dir of any python dependencies - ======================= ============================= +===================== ==================================================== +``PATH`` Set to point to ``/bin`` directories of dependencies +``CMAKE_PREFIX_PATH`` Path to dependency prefixes for CMake +``PKG_CONFIG_PATH`` Path to any pkgconfig directories for dependencies +``PYTHONPATH`` Path to site-packages dir of any python dependencies +===================== ==================================================== ``PATH`` is set up to point to dependencies ``/bin`` directories so that you can use tools installed by dependency packages at build time. @@ -1993,7 +1951,7 @@ For example, ``$MPICH_ROOT/bin/mpicc`` is frequently used by dependencies of where ``cmake`` will search for dependency libraries and headers. This causes all standard CMake find commands to look in the paths of your dependencies, so you *do not* have to manually specify arguments -like ``-D DEPENDENCY_DIR=/path/to/dependency`` to ``cmake``. More on +like ``-DDEPENDENCY_DIR=/path/to/dependency`` to ``cmake``. More on this is `in the CMake documentation `_. ``PKG_CONFIG_PATH`` is for packages that attempt to discover @@ -2031,13 +1989,15 @@ flags to the compile line so that dependencies can be easily found. These flags are added for each dependency, if they exist: Compile-time library search paths - * ``-L$dep_prefix/lib`` - * ``-L$dep_prefix/lib64`` +* ``-L$dep_prefix/lib`` +* ``-L$dep_prefix/lib64`` + Runtime library search paths (RPATHs) - * ``$rpath_flag$dep_prefix/lib`` - * ``$rpath_flag$dep_prefix/lib64`` +* ``$rpath_flag$dep_prefix/lib`` +* ``$rpath_flag$dep_prefix/lib64`` + Include search paths - * ``-I$dep_prefix/include`` +* ``-I$dep_prefix/include`` An example of this would be the ``libdwarf`` build, which has one dependency: ``libelf``. Every call to ``cc`` in the ``libdwarf`` @@ -2083,7 +2043,7 @@ Compiler flags In rare circumstances such as compiling and running small unit tests, a package developer may need to know what are the appropriate compiler flags to enable features like ``OpenMP``, ``c++11``, ``c++14`` and alike. To that end the -compiler classes in ``spack`` implement the following _properties_ : +compiler classes in ``spack`` implement the following **properties**: ``openmp_flag``, ``cxx11_flag``, ``cxx14_flag``, which can be accessed in a package by ``self.compiler.cxx11_flag`` and alike. Note that the implementation is such that if a given compiler version does not support this feature, an @@ -2102,7 +2062,7 @@ Message Parsing Interface (MPI) It is common for high performance computing software/packages to use ``MPI``. As a result of conretization, a given package can be built using different implementations of MPI such as ``Openmpi``, ``MPICH`` or ``IntelMPI``. -In some scenarios to configure a package one have to provide it with appropriate MPI +In some scenarios, to configure a package, one has to provide it with appropriate MPI compiler wrappers such as ``mpicc``, ``mpic++``. However different implementations of ``MPI`` may have different names for those wrappers. In order to make package's ``install()`` method indifferent to the @@ -2119,9 +2079,9 @@ Blas and Lapack libraries Different packages provide implementation of ``Blas`` and ``Lapack`` routines. The names of the resulting static and/or shared libraries differ from package -to package. In order to make ``install()`` method indifferent to the +to package. In order to make the ``install()`` method indifferent to the choice of ``Blas`` implementation, each package which provides it -sets up ``self.spec.blas_shared_lib`` and ``self.spec.blas_static_lib `` to +sets up ``self.spec.blas_shared_lib`` and ``self.spec.blas_static_lib`` to point to the shared and static ``Blas`` libraries, respectively. The same applies to packages which provide ``Lapack``. Package developers are advised to use these variables, for example ``spec['blas'].blas_shared_lib`` instead of @@ -2241,9 +2201,14 @@ do that, e.g.: .. code-block:: python + configure_args = [ + '--prefix={0}'.format(prefix) + ] + if spec.satisfies('@1.2:1.4'): configure_args.append("CXXFLAGS='-DWITH_FEATURE'") - configure('--prefix=' + prefix, *configure_args) + + configure(*configure_args) This works for compilers, too: @@ -2306,18 +2271,18 @@ of your dependencies. You can do that by sub-scripting the spec: .. code-block:: python - my_mpi = spec['mpich'] + my_mpi = spec['mpi'] The value in the brackets needs to be some package name, and spec needs to depend on that package, or the operation will fail. For example, the above code will fail if the ``spec`` doesn't depend on -``mpich``. The value returned and assigned to ``my_mpi``, is itself +``mpi``. The value returned and assigned to ``my_mpi``, is itself just another ``Spec`` object, so you can do all the same things you would do with the package's own spec: .. code-block:: python - mpicc = new_path(my_mpi.prefix.bin, 'mpicc') + mpicc = join_path(my_mpi.prefix.bin, 'mpicc') .. _multimethods: @@ -2330,7 +2295,7 @@ packages, based on whether the package's spec satisfies particular criteria. The ``@when`` annotation lets packages declare multiple versions of -methods like install() that depend on the package's spec. For +methods like ``install()`` that depend on the package's spec. For example: .. code-block:: python @@ -2341,16 +2306,17 @@ example: def install(self, prefix): # Do default install - @when('arch=linux-debian7-x86_64') + @when('arch=chaos_5_x86_64_ib') def install(self, prefix): # This will be executed instead of the default install if # the package's sys_type() is chaos_5_x86_64_ib. - @when('arch=linux-debian7-x86_64") + @when('arch=linux-debian7-x86_64') def install(self, prefix): - # This will be executed if the package's sys_type is bgqos_0 + # This will be executed if the package's sys_type() is + # linux-debian7-x86_64. -In the above code there are three versions of install(), two of which +In the above code there are three versions of ``install()``, two of which are specialized for particular platforms. The version that is called depends on the architecture of the package spec. @@ -2428,17 +2394,9 @@ Shell command functions Recall the install method from ``libelf``: -.. code-block:: python - - def install(self, spec, prefix): - configure("--prefix=" + prefix, - "--enable-shared", - "--disable-dependency-tracking", - "--disable-debug") - make() - - # The mkdir commands in libelf's install can fail in parallel - make("install", parallel=False) +.. literalinclude:: ../../../var/spack/repos/builtin/packages/libelf/package.py + :pyobject: Libelf.install + :linenos: Normally in Python, you'd have to write something like this in order to execute shell commands: @@ -2446,7 +2404,7 @@ to execute shell commands: .. code-block:: python import subprocess - subprocess.check_call('configure', '--prefix=' + prefix) + subprocess.check_call('configure', '--prefix={0}'.format(prefix)) We've tried to make this a bit easier by providing callable wrapper objects for some shell commands. By default, ``configure``, @@ -2488,7 +2446,7 @@ Consider a simple autotools build like this: .. code-block:: python def install(self, spec, prefix): - configure("--prefix=" + prefix) + configure("--prefix={0}".format(prefix)) make() make("install") @@ -2525,7 +2483,7 @@ Now, after ``install()`` runs, Spack will check whether ``$prefix/include/libelf.h`` exists and is a file, and whether ``$prefix/lib`` exists and is a directory. If the checks fail, then the build will fail and the install prefix will be removed. If they -succeed, Spack considers the build succeeful and keeps the prefix in +succeed, Spack considers the build successful and keeps the prefix in place. .. _file-manipulation: @@ -2633,8 +2591,8 @@ File functions install('my-header.h', join_path(prefix.include)) -:py:func:`join_path(prefix, *args) ` Like - ``os.path.join``, this joins paths using the OS path separator. +:py:func:`join_path(prefix, *args) ` + Like ``os.path.join``, this joins paths using the OS path separator. However, this version allows an arbitrary number of arguments, so you can string together many path components. @@ -2708,9 +2666,9 @@ variant names are: Name Default Description ======= ======== ======================== shared True Build shared libraries - static Build static libraries - mpi Use MPI - python Build Python extension + static True Build static libraries + mpi True Use MPI + python False Build Python extension ======= ======== ======================== If specified in this table, the corresponding default should be used @@ -2720,7 +2678,7 @@ when declaring a variant. Version Lists ^^^^^^^^^^^^^ -Spack packges should list supported versions with the newest first. +Spack packages should list supported versions with the newest first. ^^^^^^^^^^^^^^^^ Special Versions @@ -2728,20 +2686,30 @@ Special Versions The following *special* version names may be used when building a package: -* *@system*: Indicates a hook to the OS-installed version of the - package. This is useful, for example, to tell Spack to use the - OS-installed version in ``packages.yaml``:: +""""""""""" +``@system`` +""""""""""" + +Indicates a hook to the OS-installed version of the +package. This is useful, for example, to tell Spack to use the +OS-installed version in ``packages.yaml``: - openssl: - paths: - openssl@system: /usr - buildable: False +.. code-block:: yaml - Certain Spack internals look for the *@system* version and do - appropriate things in that case. + openssl: + paths: + openssl@system: /usr + buildable: False -* *@local*: Indicates the version was built manually from some source - tree of unknown provenance (see ``spack setup``). +Certain Spack internals look for the ``@system`` version and do +appropriate things in that case. + +"""""""""" +``@local`` +"""""""""" + +Indicates the version was built manually from some source +tree of unknown provenance (see ``spack setup``). --------------------------- Packaging workflow commands @@ -2760,7 +2728,7 @@ of the build. A typical package workflow might look like this: -.. code-block:: sh +.. code-block:: console $ spack edit mypackage $ spack install mypackage @@ -2823,11 +2791,11 @@ Restores the source code to pristine state, as it was before building. Does this in one of two ways: - 1. If the source was fetched as a tarball, deletes the entire build - directory and re-expands the tarball. +#. If the source was fetched as a tarball, deletes the entire build + directory and re-expands the tarball. - 2. If the source was checked out from a repository, this deletes the - build directory and checks it out again. +#. If the source was checked out from a repository, this deletes the + build directory and checks it out again. .. _spack-clean: @@ -2865,9 +2833,9 @@ By default, ``spack install`` will delete the staging area once a package has been successfully built and installed. Use ``--keep-stage`` to leave the build directory intact: -.. code-block:: sh +.. code-block:: console - spack install --keep-stage + $ spack install --keep-stage This allows you to inspect the build directory and potentially debug the build. You can use ``purge`` or ``clean`` later to get rid of the @@ -2882,17 +2850,17 @@ install prefix if anything fails during ``install()``. If you want to keep the prefix anyway (e.g. to diagnose a bug), you can use ``--keep-prefix``: -.. code-block:: sh +.. code-block:: console - spack install --keep-prefix + $ spack install --keep-prefix Note that this may confuse Spack into thinking that the package has -been installed properly, so you may need to use ``spack uninstall -f`` +been installed properly, so you may need to use ``spack uninstall --force`` to get rid of the install prefix before you build again: -.. code-block:: sh +.. code-block:: console - spack uninstall -f + $ spack uninstall --force --------------------- Graphing dependencies @@ -2906,31 +2874,9 @@ Graphing dependencies Spack provides the ``spack graph`` command for graphing dependencies. The command by default generates an ASCII rendering of a spec's -dependency graph. For example:: - - $ spack graph mpileaks - o mpileaks - |\ - | |\ - | o | callpath - |/| | - | |\| - | |\ \ - | | |\ \ - | | | | o adept-utils - | |_|_|/| - |/| | | | - o | | | | mpi - / / / / - | | o | dyninst - | |/| | - |/|/| | - | | |/ - | o | libdwarf - |/ / - o | libelf - / - o boost +dependency graph. For example: + +.. command-output:: spack graph mpileaks At the top is the root package in the DAG, with dependency edges emerging from it. On a color terminal, the edges are colored by which @@ -2938,39 +2884,9 @@ dependency they lead to. You can also use ``spack graph`` to generate graphs in the widely used `Dot `_ format. For -example:: - - $ spack graph --dot mpileaks - digraph G { - label = "Spack Dependencies" - labelloc = "b" - rankdir = "LR" - ranksep = "5" - - "boost" [label="boost"] - "callpath" [label="callpath"] - "libdwarf" [label="libdwarf"] - "mpileaks" [label="mpileaks"] - "mpi" [label="mpi"] - "adept-utils" [label="adept-utils"] - "dyninst" [label="dyninst"] - "libelf" [label="libelf"] - - "callpath" -> "dyninst" - "callpath" -> "adept-utils" - "callpath" -> "mpi" - "callpath" -> "libelf" - "callpath" -> "libdwarf" - "libdwarf" -> "libelf" - "mpileaks" -> "adept-utils" - "mpileaks" -> "callpath" - "mpileaks" -> "mpi" - "adept-utils" -> "boost" - "adept-utils" -> "mpi" - "dyninst" -> "boost" - "dyninst" -> "libelf" - "dyninst" -> "libdwarf" - } +example: + +.. command-output:: spack graph --dot mpileaks This graph can be provided as input to other graphing tools, such as those in `Graphviz `_. @@ -2981,14 +2897,19 @@ Interactive shell support Spack provides some limited shell support to make life easier for packagers. You can enable these commands by sourcing a setup file in -the ``/share/spack`` directory. For ``bash`` or ``ksh``, run:: +the ``share/spack`` directory. For ``bash`` or ``ksh``, run: - . $SPACK_ROOT/share/spack/setup-env.sh +.. code-block:: sh + + export SPACK_ROOT=/path/to/spack + . $SPACK_ROOT/share/spack/setup-env.sh For ``csh`` and ``tcsh`` run: - setenv SPACK_ROOT /path/to/spack - source $SPACK_ROOT/share/spack/setup-env.csh +.. code-block:: csh + + setenv SPACK_ROOT /path/to/spack + source $SPACK_ROOT/share/spack/setup-env.csh ``spack cd`` will then be available. @@ -3002,7 +2923,7 @@ For ``csh`` and ``tcsh`` run: Suppose you've staged a package but you want to modify it before you build it: -.. code-block:: sh +.. code-block:: console $ spack stage libelf ==> Trying to fetch from http://www.mr511.de/software/libelf-0.8.13.tar.gz @@ -3013,7 +2934,7 @@ build it: $ pwd /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-debian7-x86_64/libelf-0.8.13 -``spack cd`` here changed he current working directory to the +``spack cd`` here changed the current working directory to the directory containing the expanded ``libelf`` source code. There are a number of other places you can cd to in the spack directory hierarchy: @@ -3035,16 +2956,17 @@ but it takes a spec as an argument. You can use it to see the environment variables that will be set when a particular build runs, for example: -.. code-block:: sh +.. code-block:: console $ spack env mpileaks@1.1%intel This will display the entire environment that will be set when the ``mpileaks@1.1%intel`` build runs. -To run commands in a package's build environment, you can simply provided them after the spec argument to ``spack env``: +To run commands in a package's build environment, you can simply +provide them after the spec argument to ``spack env``: -.. code-block:: sh +.. code-block:: console $ spack cd mpileaks@1.1%intel $ spack env mpileaks@1.1%intel ./configure @@ -3060,13 +2982,17 @@ package's build environment. ``spack location`` is the same as ``spack cd`` but it does not require shell support. It simply prints out the path you ask for, rather than -cd'ing to it. In bash, this:: +cd'ing to it. In bash, this: - cd $(spack location -b ) +.. code-block:: console -is the same as:: + $ cd $(spack location -b ) - spack cd -b +is the same as: + +.. code-block:: console + + $ spack cd -b ``spack location`` is intended for use in scripts or makefiles that need to know where packages are installed. e.g., in a makefile you @@ -3082,7 +3008,7 @@ might write: Build System Configuration Support ---------------------------------- -Imagine a developer creating a CMake-based (or Autotools) project in a local +Imagine a developer creating a CMake or Autotools-based project in a local directory, which depends on libraries A-Z. Once Spack has installed those dependencies, one would like to run ``cmake`` with appropriate command line and environment so CMake can find them. The ``spack @@ -3091,44 +3017,45 @@ configuration that is essentially the same as how Spack *would have* configured the project. This can be demonstrated with a usage example: -.. code-block:: bash +.. code-block:: console - cd myproject - spack setup myproject@local - mkdir build; cd build - ../spconfig.py .. - make - make install + $ cd myproject + $ spack setup myproject@local + $ mkdir build; cd build + $ ../spconfig.py .. + $ make + $ make install Notes: - * Spack must have ``myproject/package.py`` in its repository for - this to work. - * ``spack setup`` produces the executable script ``spconfig.py`` in - the local directory, and also creates the module file for the - package. ``spconfig.py`` is normally run from the user's - out-of-source build directory. - * The version number given to ``spack setup`` is arbitrary, just - like ``spack diy``. ``myproject/package.py`` does not need to - have any valid downloadable versions listed (typical when a - project is new). - * spconfig.py produces a CMake configuration that *does not* use the - Spack wrappers. Any resulting binaries *will not* use RPATH, - unless the user has enabled it. This is recommended for - development purposes, not production. - * ``spconfig.py`` is human readable, and can serve as a developer - reference of what dependencies are being used. - * ``make install`` installs the package into the Spack repository, - where it may be used by other Spack packages. - * CMake-generated makefiles re-run CMake in some circumstances. Use - of ``spconfig.py`` breaks this behavior, requiring the developer - to manually re-run ``spconfig.py`` when a ``CMakeLists.txt`` file - has changed. + +* Spack must have ``myproject/package.py`` in its repository for + this to work. +* ``spack setup`` produces the executable script ``spconfig.py`` in + the local directory, and also creates the module file for the + package. ``spconfig.py`` is normally run from the user's + out-of-source build directory. +* The version number given to ``spack setup`` is arbitrary, just + like ``spack diy``. ``myproject/package.py`` does not need to + have any valid downloadable versions listed (typical when a + project is new). +* spconfig.py produces a CMake configuration that *does not* use the + Spack wrappers. Any resulting binaries *will not* use RPATH, + unless the user has enabled it. This is recommended for + development purposes, not production. +* ``spconfig.py`` is human readable, and can serve as a developer + reference of what dependencies are being used. +* ``make install`` installs the package into the Spack repository, + where it may be used by other Spack packages. +* CMake-generated makefiles re-run CMake in some circumstances. Use + of ``spconfig.py`` breaks this behavior, requiring the developer + to manually re-run ``spconfig.py`` when a ``CMakeLists.txt`` file + has changed. ^^^^^^^^^^^^ CMakePackage ^^^^^^^^^^^^ -In order ot enable ``spack setup`` functionality, the author of +In order to enable ``spack setup`` functionality, the author of ``myproject/package.py`` must subclass from ``CMakePackage`` instead of the standard ``Package`` superclass. Because CMake is standardized, the packager does not need to tell Spack how to run @@ -3140,14 +3067,15 @@ translate variant flags into CMake definitions. For example: .. code-block:: python - def configure_args(self): - spec = self.spec - return [ - '-DUSE_EVERYTRACE=%s' % ('YES' if '+everytrace' in spec else 'NO'), - '-DBUILD_PYTHON=%s' % ('YES' if '+python' in spec else 'NO'), - '-DBUILD_GRIDGEN=%s' % ('YES' if '+gridgen' in spec else 'NO'), - '-DBUILD_COUPLER=%s' % ('YES' if '+coupler' in spec else 'NO'), - '-DUSE_PISM=%s' % ('YES' if '+pism' in spec else 'NO')] + def configure_args(self): + spec = self.spec + return [ + '-DUSE_EVERYTRACE=%s' % ('YES' if '+everytrace' in spec else 'NO'), + '-DBUILD_PYTHON=%s' % ('YES' if '+python' in spec else 'NO'), + '-DBUILD_GRIDGEN=%s' % ('YES' if '+gridgen' in spec else 'NO'), + '-DBUILD_COUPLER=%s' % ('YES' if '+coupler' in spec else 'NO'), + '-DUSE_PISM=%s' % ('YES' if '+pism' in spec else 'NO') + ] If needed, a packager may also override methods defined in ``StagedPackage`` (see below). -- cgit v1.2.3-70-g09d2