From 74dc7ffe4d9824f916c37b151e9ebe63a184c705 Mon Sep 17 00:00:00 2001 From: Gregory Becker Date: Wed, 25 May 2016 10:56:12 -0700 Subject: updated documentation for cflags PR --- lib/spack/docs/basic_usage.rst | 155 +++++++++++++++++++++++++++---------- lib/spack/docs/configuration.rst | 12 +-- lib/spack/docs/features.rst | 17 ++-- lib/spack/docs/packaging_guide.rst | 48 +++++++----- 4 files changed, 158 insertions(+), 74 deletions(-) (limited to 'lib') diff --git a/lib/spack/docs/basic_usage.rst b/lib/spack/docs/basic_usage.rst index 6efed83621..2eed9dddd4 100644 --- a/lib/spack/docs/basic_usage.rst +++ b/lib/spack/docs/basic_usage.rst @@ -102,8 +102,8 @@ that the packages is installed: ==> adept-utils is already installed in /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/adept-utils@1.0-5adef8da. ==> Trying to fetch from https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz ######################################################################## 100.0% - ==> Staging archive: /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7=chaos_5_x86_64_ib-59f6ad23/mpileaks-1.0.tar.gz - ==> Created stage in /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7=chaos_5_x86_64_ib-59f6ad23. + ==> Staging archive: /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=chaos_5_x86_64_ib-59f6ad23/mpileaks-1.0.tar.gz + ==> Created stage in /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=chaos_5_x86_64_ib-59f6ad23. ==> No patches needed for mpileaks. ==> Building mpileaks. @@ -132,10 +132,10 @@ sites, as installing a version that one user needs will not disrupt existing installations for other users. In addition to different versions, Spack can customize the compiler, -compile-time options (variants), and platform (for cross compiles) of -an installation. Spack is unique in that it can also configure the -*dependencies* a package is built with. For example, two -configurations of the same version of a package, one built with boost +compile-time options (variants), compiler flags, and platform (for +cross compiles) of an installation. Spack is unique in that it can +also configure the *dependencies* a package is built with. For example, +two configurations of the same version of a package, one built with boost 1.39.0, and the other version built with version 1.43.0, can coexist. This can all be done on the command line using the *spec* syntax. @@ -334,6 +334,11 @@ of libelf would look like this: -- chaos_5_x86_64_ib / gcc@4.4.7 -------------------------------- libdwarf@20130729-d9b90962 +We can also search for packages that have a certain attribute. For example, +``spack find -l libdwarf +debug`` will show only installations of libdwarf +with the 'debug' compile-time option enabled, while ``spack find -l +debug`` +will find every installed package with a 'debug' compile-time option enabled. + The full spec syntax is discussed in detail in :ref:`sec-specs`. @@ -463,6 +468,26 @@ For compilers, like ``clang``, that do not support Fortran, put Once you save the file, the configured compilers will show up in the list displayed by ``spack compilers``. +You can also add compiler flags to manually configured compilers. The +valid flags are ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, +``ldflags``, and ``ldlibs``. For example,:: + + ... + chaos_5_x86_64_ib: + ... + intel@15.0.0: + cc: /usr/local/bin/icc-15.0.024-beta + cxx: /usr/local/bin/icpc-15.0.024-beta + f77: /usr/local/bin/ifort-15.0.024-beta + fc: /usr/local/bin/ifort-15.0.024-beta + cppflags: -O3 -fPIC + ... + +These flags will be treated by spack as if they were enterred from +the command line each time this compiler is used. The compiler wrappers +then inject those flags into the compiler command. Compiler flags +enterred from the command line will be discussed in more detail in the +following section. .. _sec-specs: @@ -480,7 +505,7 @@ the full syntax of specs. Here is an example of a much longer spec than we've seen thus far:: - mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt =bgqos_0 ^callpath @1.1 %gcc@4.7.2 + mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt arch=bgq_os ^callpath @1.1 %gcc@4.7.2 If provided to ``spack install``, this will install the ``mpileaks`` library at some version between ``1.2`` and ``1.4`` (inclusive), @@ -498,8 +523,12 @@ More formally, a spec consists of the following pieces: * ``%`` Optional compiler specifier, with an optional compiler version (``gcc`` or ``gcc@4.7.3``) * ``+`` or ``-`` or ``~`` Optional variant specifiers (``+debug``, - ``-qt``, or ``~qt``) -* ``=`` Optional architecture specifier (``bgqos_0``) + ``-qt``, or ``~qt``) for boolean variants +* ``name=`` Optional variant specifiers that are not restricted to +boolean variants +* ``name=`` Optional compiler flag specifiers. Valid flag names are +``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, ``ldflags``, and ``ldlibs``. +* ``arch=`` Optional architecture specifier (``arch=bgq_os``) * ``^`` Dependency specs (``^callpath@1.1``) There are two things to notice here. The first is that specs are @@ -579,7 +608,7 @@ compilers, variants, and architectures just like any other spec. Specifiers are associated with the nearest package name to their left. For example, above, ``@1.1`` and ``%gcc@4.7.2`` associates with the ``callpath`` package, while ``@1.2:1.4``, ``%gcc@4.7.5``, ``+debug``, -``-qt``, and ``=bgqos_0`` all associate with the ``mpileaks`` package. +``-qt``, and ``arch=bgq_os`` all associate with the ``mpileaks`` package. In the diagram above, ``mpileaks`` depends on ``mpich`` with an unspecified version, but packages can depend on other packages with @@ -635,22 +664,25 @@ based on site policies. Variants ~~~~~~~~~~~~~~~~~~~~~~~ -.. Note:: - - Variants are not yet supported, but will be in the next Spack - release (0.9), due in Q2 2015. - -Variants are named options associated with a particular package, and -they can be turned on or off. For example, above, supplying -``+debug`` causes ``mpileaks`` to be built with debug flags. The -names of particular variants available for a package depend on what -was provided by the package author. ``spack info `` will +Variants are named options associated with a particular package. They are +optional, as each package must provide default values for each variant it +makes available. Variants can be specified using +a flexible parameter syntax ``name=``. For example, +``spack install libelf debug=True`` will install libelf build with debug +flags. The names of particular variants available for a package depend on +what was provided by the package author. ``spack into `` will provide information on what build variants are available. -Depending on the package a variant may be on or off by default. For -``mpileaks`` here, ``debug`` is off by default, and we turned it on -with ``+debug``. If a package is on by default you can turn it off by -either adding ``-name`` or ``~name`` to the spec. +For compatibility with earlier versions, variants which happen to be +boolean in nature can be specified by a syntax that represents turning +options on and off. For example, in the previous spec we could have +supplied ``libelf +debug`` with the same effect of enabling the debug +compile time option for the libelf package. + +Depending on the package a variant may have any default value. For +``libelf`` here, ``debug`` is ``False`` by default, and we turned it on +with ``debug=True`` or ``+debug``. If a package is ``True`` by default +you can turn it off by either adding ``-name`` or ``~name`` to the spec. There are two syntaxes here because, depending on context, ``~`` and ``-`` may mean different things. In most shells, the following will @@ -662,7 +694,7 @@ result in the shell performing home directory substitution: mpileaks~debug # use this instead If there is a user called ``debug``, the ``~`` will be incorrectly -expanded. In this situation, you would want to write ``mpileaks +expanded. In this situation, you would want to write ``libelf -debug``. However, ``-`` can be ambiguous when included after a package name without spaces: @@ -677,12 +709,35 @@ package, not a request for ``mpileaks`` built without ``debug`` options. In this scenario, you should write ``mpileaks~debug`` to avoid ambiguity. -When spack normalizes specs, it prints them out with no spaces and -uses only ``~`` for disabled variants. We allow ``-`` and spaces on -the command line is provided for convenience and legibility. +When spack normalizes specs, it prints them out with no spaces boolean +variants using the backwards compatibility syntax and uses only ``~`` +for disabled boolean variants. We allow ``-`` and spaces on the command +line is provided for convenience and legibility. -Architecture specifier +Compiler Flags +~~~~~~~~~~~~~~~~~~~~~~~ + +Compiler flags are specified using the same syntax as non-boolean variants, +but fulfill a different purpose. While the function of a variant is set by +the package, compiler flags are used by the compiler wrappers to inject +flags into the compile line of the build. Additionally, compiler flags are +inherited by dependencies. ``spack install libdwarf cppflags=\"-g\"`` will +install both libdwarf and libelf with the ``-g`` flag injected into their +compile line. + +Notice that the value of the compiler flags must be escape quoted on the +command line. From within python files, the same spec would be specified +``libdwarf cppflags="-g"``. This is necessary because of how the shell +handles the quote symbols. + +The six compiler flags are injected in the order of implicit make commands +in gnu autotools. If all flags are set, the order is +``$cppflags $cflags|$cxxflags $ldflags command $ldlibs`` for C and C++ and +``$fflags $cppflags $ldflags command $ldlibs`` for fortran. + + +Architecture specifiers ~~~~~~~~~~~~~~~~~~~~~~~ .. Note:: @@ -690,12 +745,9 @@ Architecture specifier Architecture specifiers are part of specs but are not yet functional. They will be in Spack version 1.0, due in Q3 2015. -The architecture specifier starts with a ``=`` and also comes after -some package name within a spec. It allows a user to specify a -particular architecture for the package to be built. This is mostly -used for architectures that need cross-compilation, and in most cases, -users will not need to specify the architecture when they install a -package. +The architecture specifier looks identical to a variant specifier for a +non-boolean variant. The architecture can be specified only using the +reserved name ``arch`` (``arch=bgq_os``). .. _sec-virtual-dependencies: @@ -773,6 +825,23 @@ any MPI implementation will do. If another package depends on error. Likewise, if you try to plug in some package that doesn't provide MPI, Spack will raise an error. +Specifying Specs by Hash +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Complicated specs can become cumbersome to enter on the command line, +especially when many of the qualifications are necessary to +distinguish between similar installs, for example when using the +``uninstall`` command. To avoid this, when referencing an existing spec, +Spack allows you to reference specs by their hash. We previously +discussed the spec hash that Spack computes. In place of a spec in any +command, substitute ``/`` where ```` is any amount from +the beginning of a spec hash. If the given spec hash is sufficient +to be unique, Spack will replace the reference with the spec to which +it refers. Otherwise, it will prompt for a more qualified hash. + +Note that this will not work to reinstall a depencency uninstalled by +``spack uninstall -f``. + .. _spack-providers: ``spack providers`` @@ -1002,8 +1071,8 @@ than one installed package matches it), then Spack will warn you: $ spack load libelf ==> Error: Multiple matches for spec libelf. Choose one: - libelf@0.8.13%gcc@4.4.7=chaos_5_x86_64_ib - libelf@0.8.13%intel@15.0.0=chaos_5_x86_64_ib + libelf@0.8.13%gcc@4.4.7 arch=chaos_5_x86_64_ib + libelf@0.8.13%intel@15.0.0 arch=chaos_5_x86_64_ib You can either type the ``spack load`` command again with a fully qualified argument, or you can add just enough extra constraints to @@ -1282,7 +1351,7 @@ You can find extensions for your Python installation like this: .. code-block:: sh $ spack extensions python - ==> python@2.7.8%gcc@4.4.7=chaos_5_x86_64_ib-703c7a96 + ==> python@2.7.8%gcc@4.4.7 arch=chaos_5_x86_64_ib-703c7a96 ==> 36 extensions: geos py-ipython py-pexpect py-pyside py-sip py-basemap py-libxml2 py-pil py-pytz py-six @@ -1372,9 +1441,9 @@ installation: .. code-block:: sh $ spack activate py-numpy - ==> Activated extension py-setuptools@11.3.1%gcc@4.4.7=chaos_5_x86_64_ib-3c74eb69 for python@2.7.8%gcc@4.4.7. - ==> Activated extension py-nose@1.3.4%gcc@4.4.7=chaos_5_x86_64_ib-5f70f816 for python@2.7.8%gcc@4.4.7. - ==> Activated extension py-numpy@1.9.1%gcc@4.4.7=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7. + ==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-3c74eb69 for python@2.7.8%gcc@4.4.7. + ==> Activated extension py-nose@1.3.4%gcc@4.4.7 arch=chaos_5_x86_64_ib-5f70f816 for python@2.7.8%gcc@4.4.7. + ==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7. Several things have happened here. The user requested that ``py-numpy`` be activated in the ``python`` installation it was built @@ -1389,7 +1458,7 @@ packages listed as activated: .. code-block:: sh $ spack extensions python - ==> python@2.7.8%gcc@4.4.7=chaos_5_x86_64_ib-703c7a96 + ==> python@2.7.8%gcc@4.4.7 arch=chaos_5_x86_64_ib-703c7a96 ==> 36 extensions: geos py-ipython py-pexpect py-pyside py-sip py-basemap py-libxml2 py-pil py-pytz py-six @@ -1437,7 +1506,7 @@ dependencies, you can use ``spack activate -f``: .. code-block:: sh $ spack activate -f py-numpy - ==> Activated extension py-numpy@1.9.1%gcc@4.4.7=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7. + ==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7. .. _spack-deactivate: diff --git a/lib/spack/docs/configuration.rst b/lib/spack/docs/configuration.rst index c0b79a7f44..c613071c65 100644 --- a/lib/spack/docs/configuration.rst +++ b/lib/spack/docs/configuration.rst @@ -70,9 +70,9 @@ directory. Here's an example of an external configuration: packages: openmpi: paths: - openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib: /opt/openmpi-1.4.3 - openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug - openmpi@1.6.5%intel@10.1=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel + openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib: /opt/openmpi-1.4.3 + openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug + openmpi@1.6.5%intel@10.1 arch=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel This example lists three installations of OpenMPI, one built with gcc, one built with gcc and debug information, and another built with Intel. @@ -108,9 +108,9 @@ be: packages: openmpi: paths: - openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib: /opt/openmpi-1.4.3 - openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug - openmpi@1.6.5%intel@10.1=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel + openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib: /opt/openmpi-1.4.3 + openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug + openmpi@1.6.5%intel@10.1 arch=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel buildable: False The addition of the ``buildable`` flag tells Spack that it should never build diff --git a/lib/spack/docs/features.rst b/lib/spack/docs/features.rst index 0998ba8da4..27a3b4b435 100644 --- a/lib/spack/docs/features.rst +++ b/lib/spack/docs/features.rst @@ -31,14 +31,21 @@ platform, all on the command line. # Specify a compiler (and its version), with % $ spack install mpileaks@1.1.2 %gcc@4.7.3 - # Add special compile-time options with + + # Add special compile-time options by name + $ spack install mpileaks@1.1.2 %gcc@4.7.3 debug=True + + # Add special boolean compile-time options with + $ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug - # Cross-compile for a different architecture with = - $ spack install mpileaks@1.1.2 =bgqos_0 + # Add compiler flags using the conventional names + $ spack install mpileaks@1.1.2 %gcc@4.7.3 cppflags=\"-O3 -floop-block\" + + # Cross-compile for a different architecture with arch= + $ spack install mpileaks@1.1.2 arch=bgqos_0 -Users can specify as many or few options as they care about. Spack -will fill in the unspecified values with sensible defaults. +Users can specify as many or few options as they care about. Spack +will fill in the unspecified values with sensible defaults. The two listed +syntaxes for variants are identical when the value is boolean. Customize dependencies diff --git a/lib/spack/docs/packaging_guide.rst b/lib/spack/docs/packaging_guide.rst index 63c411ffb5..1f83f611b0 100644 --- a/lib/spack/docs/packaging_guide.rst +++ b/lib/spack/docs/packaging_guide.rst @@ -1221,11 +1221,13 @@ just as easily provide a version range: depends_on("libelf@0.8.2:0.8.4:") -Or a requirement for a particular variant: +Or a requirement for a particular variant or compiler flags: .. code-block:: python depends_on("libelf@0.8+debug") + depends_on('libelf debug=True') + depends_on('libelf cppflags="-fPIC") Both users *and* package authors can use the same spec syntax to refer to different package configurations. Users use the spec syntax on the @@ -1623,21 +1625,21 @@ the user runs ``spack install`` and the time the ``install()`` method is called. The concretized version of the spec above might look like this:: - mpileaks@2.3%gcc@4.7.3=linux-ppc64 - ^callpath@1.0%gcc@4.7.3+debug=linux-ppc64 - ^dyninst@8.1.2%gcc@4.7.3=linux-ppc64 - ^libdwarf@20130729%gcc@4.7.3=linux-ppc64 - ^libelf@0.8.11%gcc@4.7.3=linux-ppc64 - ^mpich@3.0.4%gcc@4.7.3=linux-ppc64 + mpileaks@2.3%gcc@4.7.3 arch=linux-ppc64 + ^callpath@1.0%gcc@4.7.3+debug arch=linux-ppc64 + ^dyninst@8.1.2%gcc@4.7.3 arch=linux-ppc64 + ^libdwarf@20130729%gcc@4.7.3 arch=linux-ppc64 + ^libelf@0.8.11%gcc@4.7.3 arch=linux-ppc64 + ^mpich@3.0.4%gcc@4.7.3 arch=linux-ppc64 .. graphviz:: digraph { - "mpileaks@2.3\n%gcc@4.7.3\n=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n=linux-ppc64" - "mpileaks@2.3\n%gcc@4.7.3\n=linux-ppc64" -> "callpath@1.0\n%gcc@4.7.3+debug\n=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n=linux-ppc64" - "callpath@1.0\n%gcc@4.7.3+debug\n=linux-ppc64" -> "dyninst@8.1.2\n%gcc@4.7.3\n=linux-ppc64" - "dyninst@8.1.2\n%gcc@4.7.3\n=linux-ppc64" -> "libdwarf@20130729\n%gcc@4.7.3\n=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n=linux-ppc64" - "dyninst@8.1.2\n%gcc@4.7.3\n=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n=linux-ppc64" + "mpileaks@2.3\n%gcc@4.7.3\n arch=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n arch=linux-ppc64" + "mpileaks@2.3\n%gcc@4.7.3\n arch=linux-ppc64" -> "callpath@1.0\n%gcc@4.7.3+debug\n arch=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n arch=linux-ppc64" + "callpath@1.0\n%gcc@4.7.3+debug\n arch=linux-ppc64" -> "dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64" + "dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64" -> "libdwarf@20130729\n%gcc@4.7.3\n arch=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n arch=linux-ppc64" + "dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n arch=linux-ppc64" } Here, all versions, compilers, and platforms are filled in, and there @@ -1666,9 +1668,9 @@ running ``spack spec``. For example: ^libdwarf ^libelf - dyninst@8.0.1%gcc@4.7.3=linux-ppc64 - ^libdwarf@20130729%gcc@4.7.3=linux-ppc64 - ^libelf@0.8.13%gcc@4.7.3=linux-ppc64 + dyninst@8.0.1%gcc@4.7.3 arch=linux-ppc64 + ^libdwarf@20130729%gcc@4.7.3 arch=linux-ppc64 + ^libelf@0.8.13%gcc@4.7.3 arch=linux-ppc64 This is useful when you want to know exactly what Spack will do when you ask for a particular spec. @@ -1908,6 +1910,12 @@ the command line. ``$rpath_flag`` can be overriden on a compiler specific basis in ``lib/spack/spack/compilers/$compiler.py``. +The compiler wrappers also pass the compiler flags specified by the user from +the command line (``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, ``ldflags``, +and/or ``ldlibs``). They do not override the canonical autotools flags with the +same names (but in ALL-CAPS) that may be passed into the build by particularly +challenging package scripts. + Compiler flags ~~~~~~~~~~~~~~ In rare circumstances such as compiling and running small unit tests, a package @@ -2154,12 +2162,12 @@ example: def install(self, prefix): # Do default install - @when('=chaos_5_x86_64_ib') + @when('arch=chaos_5_x86_64_ib') def install(self, prefix): # This will be executed instead of the default install if # the package's sys_type() is chaos_5_x86_64_ib. - @when('=bgqos_0") + @when('arch=bgqos_0") def install(self, prefix): # This will be executed if the package's sys_type is bgqos_0 @@ -2749,11 +2757,11 @@ build it: $ spack stage libelf ==> Trying to fetch from http://www.mr511.de/software/libelf-0.8.13.tar.gz ######################################################################## 100.0% - ==> Staging archive: /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3=linux-ppc64/libelf-0.8.13.tar.gz - ==> Created stage in /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3=linux-ppc64. + ==> Staging archive: /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64/libelf-0.8.13.tar.gz + ==> Created stage in /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64. $ spack cd libelf $ pwd - /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3=linux-ppc64/libelf-0.8.13 + /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64/libelf-0.8.13 ``spack cd`` here changed he current working directory to the directory containing the expanded ``libelf`` source code. There are a -- cgit v1.2.3-60-g2f50