diff options
31 files changed, 28 insertions, 7565 deletions
diff --git a/lib/spack/docs/.gitignore b/lib/spack/docs/.gitignore index 0b6258af50..1624dfeaea 100644 --- a/lib/spack/docs/.gitignore +++ b/lib/spack/docs/.gitignore @@ -3,3 +3,5 @@ command_index.rst spack*.rst llnl*.rst _build +.spack-env +spack.lock diff --git a/lib/spack/docs/build_systems/intelpackage.rst b/lib/spack/docs/build_systems/intelpackage.rst index 30c567373e..f7d975899e 100644 --- a/lib/spack/docs/build_systems/intelpackage.rst +++ b/lib/spack/docs/build_systems/intelpackage.rst @@ -120,7 +120,7 @@ version numbers seen with most other Spack packages. For example, we have: ... Preferred version: professional.2018.3 http:... - + Safe versions: professional.2018.3 http:... ... @@ -728,7 +728,7 @@ For packages that contain a compiler, follow `the previous section .. code-block:: console - $ spack install intel-mpi@2018.3.199 + $ spack install intel-mpi@2018.3.199 $ spack install intel-mpi@2018.3.199 %intel@18 4. To prepare the new packages for use with client packages, @@ -802,7 +802,7 @@ by one of the following means: Configure the order of compilers in the appropriate ``packages.yaml`` file, under either an ``all:`` or client-package-specific entry, in a ``compiler:`` list. Consult the Spack documentation for - :ref:`Configuring Package Preferences <configs-tutorial-package-prefs>` + `Configuring Package Preferences <https://spack-tutorial.readthedocs.io/en/latest/tutorial_configuration.html#configuring-package-preferences>`_ and :ref:`Concretization Preferences <concretization-preferences>`. @@ -851,7 +851,7 @@ client packages, edit the ``packages.yaml`` file. Customize, either in the the virtual packages and whose values are the Spack specs that satisfy the virtual package, in order of decreasing preference. To learn more about the ``providers:`` settings, see the Spack tutorial for -:ref:`Configuring Package Preferences <configs-tutorial-package-prefs>` +`Configuring Package Preferences <https://spack-tutorial.readthedocs.io/en/latest/tutorial_configuration.html#configuring-package-preferences>`_ and the section :ref:`Concretization Preferences <concretization-preferences>`. @@ -972,7 +972,7 @@ a *virtual* ``mkl`` package is declared in Spack. .. code-block:: python self.spec['blas'].headers.include_flags - + and to generate linker options (``-L<dir> -llibname ...``), use the same as above, .. code-block:: python diff --git a/lib/spack/docs/conf.py b/lib/spack/docs/conf.py index cacbbb3a41..e5305a9317 100644 --- a/lib/spack/docs/conf.py +++ b/lib/spack/docs/conf.py @@ -159,7 +159,7 @@ gettext_uuid = False # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. -exclude_patterns = ['_build', '_spack_root'] +exclude_patterns = ['_build', '_spack_root', '.spack-env'] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None diff --git a/lib/spack/docs/index.rst b/lib/spack/docs/index.rst index 5c93285cbb..48af67982a 100644 --- a/lib/spack/docs/index.rst +++ b/lib/spack/docs/index.rst @@ -55,7 +55,7 @@ or refer to the full manual below. getting_started basic_usage workflows - tutorial + Tutorial: Spack 101 <https://spack-tutorial.readthedocs.io> known_issues .. toctree:: diff --git a/lib/spack/docs/spack.yaml b/lib/spack/docs/spack.yaml new file mode 100644 index 0000000000..bade961fce --- /dev/null +++ b/lib/spack/docs/spack.yaml @@ -0,0 +1,19 @@ +# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other +# Spack Project Developers. See the top-level COPYRIGHT file for details. +# +# SPDX-License-Identifier: (Apache-2.0 OR MIT) + +# +# These are requirements for building the documentation. You can run +# these commands in this directory to install Sphinx and its plugins, +# then build the docs: +# +# spack install +# spack env activate . +# make +# +spack: + specs: + - py-sphinx + - py-sphinxcontrib-programoutput + - py-sphinx-rtd-theme diff --git a/lib/spack/docs/tutorial.rst b/lib/spack/docs/tutorial.rst deleted file mode 100644 index 4ece87ade0..0000000000 --- a/lib/spack/docs/tutorial.rst +++ /dev/null @@ -1,75 +0,0 @@ -.. Copyright 2013-2019 Lawrence Livermore National Security, LLC and other - Spack Project Developers. See the top-level COPYRIGHT file for details. - - SPDX-License-Identifier: (Apache-2.0 OR MIT) - -.. _spack-101: - -============================= -Tutorial: Spack 101 -============================= - -This is a full-day introduction to Spack with lectures and live demos. -It was last presented at the `Practice and Experience in Advanced -Research Computing Conference (PEARC19) -<https://www.pearc19.pearc.org/>`_ on July 31, 2019. - -You can use these materials to teach a course on Spack at your own site, -or you can just skip ahead and read the live demo scripts to see how -Spack is used in practice. - -.. _sc16-slides: - -.. rubric:: Slides - -.. figure:: tutorial/sc16-tutorial-slide-preview.png - :target: https://spack.io/slides/spack-pearc19-tutorial-slides.pdf - :height: 72px - :align: left - :alt: Slide Preview - -`Download Slides <https://spack.io/slides/spack-pearc19-tutorial-slides.pdf>`_. - -**Full citation:** Levi Baber, Gregory Becker, Adam J. Stewart, and Todd -Gamblin. Managing HPC Software Complexity with Spack. Tutorial presented -at the Practice and Experience in Advanced Research Computing Conference -(PEARC19). July 31, 2019. Chicago, IL, USA. - -.. _sc16-live-demos: - -.. rubric:: Live Demos - -We provide scripts that take you step-by-step through basic Spack tasks. -They correspond to sections in the slides above. You can use one of the -following methods to run through the scripts: - - 1. We provide the `spack/tutorial - <https://hub.docker.com/r/spack/tutorial>`_ container image on - Docker Hub that you can use to do the tutorial on your local - machine. You can invoke ``docker run -it spack/tutorial`` to start - using the container. - - 2. When we host the tutorial, we also provision VM instances in `AWS - <https://aws.amazon.com/>`_, so that users who are unfamiliar with - Docker can simply log into a VPM to do the demo exercises. - -You should now be ready to run through our demo scripts: - - 1. :ref:`basics-tutorial` - 2. :ref:`configs-tutorial` - 3. :ref:`packaging-tutorial` - 4. :ref:`environments-tutorial` - 5. :ref:`modules-tutorial` - 6. :ref:`build-systems-tutorial` - 7. :ref:`advanced-packaging-tutorial` - -Full contents: - -.. toctree:: - tutorial_basics - tutorial_configuration - tutorial_packaging - tutorial_environments - tutorial_modules - tutorial_buildsystems - tutorial_advanced_packaging diff --git a/lib/spack/docs/tutorial/examples/0.package.py b/lib/spack/docs/tutorial/examples/0.package.py deleted file mode 100644 index 990ba7d3d8..0000000000 --- a/lib/spack/docs/tutorial/examples/0.package.py +++ /dev/null @@ -1,39 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -# -# This is a template package file for Spack. We've put "FIXME" -# next to all the things you'll want to change. Once you've handled -# them, you can save this file and test your package like this: -# -# spack install mpileaks -# -# You can edit this file again by typing: -# -# spack edit mpileaks -# -# See the Spack documentation for more information on packaging. -# If you submit this package back to Spack as a pull request, -# please first remove this boilerplate and all FIXME comments. -# -from spack import * - - -class Mpileaks(Package): - """FIXME: Put a proper description of your package here.""" - - # FIXME: Add a proper url for your package's homepage here. - homepage = "http://www.example.com" - url = "https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" - - version('1.0', sha256='2e34cc4505556d1c1f085758e26f2f8eea0972db9382f051b2dcfb1d7d9e1825') - - # FIXME: Add dependencies if required. - # depends_on('foo') - - def install(self, spec, prefix): - # FIXME: Unknown build system - make() - make('install') diff --git a/lib/spack/docs/tutorial/examples/1.package.py b/lib/spack/docs/tutorial/examples/1.package.py deleted file mode 100644 index f2d515fa4a..0000000000 --- a/lib/spack/docs/tutorial/examples/1.package.py +++ /dev/null @@ -1,24 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Mpileaks(Package): - """Tool to detect and report MPI objects like MPI_Requests and - MPI_Datatypes.""" - - homepage = "https://github.com/LLNL/mpileaks" - url = "https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" - - version('1.0', sha256='2e34cc4505556d1c1f085758e26f2f8eea0972db9382f051b2dcfb1d7d9e1825') - - # FIXME: Add dependencies if required. - # depends_on('foo') - - def install(self, spec, prefix): - # FIXME: Unknown build system - make() - make('install') diff --git a/lib/spack/docs/tutorial/examples/2.package.py b/lib/spack/docs/tutorial/examples/2.package.py deleted file mode 100644 index d0f1e6d062..0000000000 --- a/lib/spack/docs/tutorial/examples/2.package.py +++ /dev/null @@ -1,25 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Mpileaks(Package): - """Tool to detect and report MPI objects like MPI_Requests and - MPI_Datatypes.""" - - homepage = "https://github.com/LLNL/mpileaks" - url = "https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" - - version('1.0', sha256='2e34cc4505556d1c1f085758e26f2f8eea0972db9382f051b2dcfb1d7d9e1825') - - depends_on('mpi') - depends_on('adept-utils') - depends_on('callpath') - - def install(self, spec, prefix): - # FIXME: Unknown build system - make() - make('install') diff --git a/lib/spack/docs/tutorial/examples/3.package.py b/lib/spack/docs/tutorial/examples/3.package.py deleted file mode 100644 index 05fd7aa00a..0000000000 --- a/lib/spack/docs/tutorial/examples/3.package.py +++ /dev/null @@ -1,25 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Mpileaks(Package): - """Tool to detect and report MPI objects like MPI_Requests and - MPI_Datatypes.""" - - homepage = "https://github.com/LLNL/mpileaks" - url = "https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" - - version('1.0', sha256='2e34cc4505556d1c1f085758e26f2f8eea0972db9382f051b2dcfb1d7d9e1825') - - depends_on('mpi') - depends_on('adept-utils') - depends_on('callpath') - - def install(self, spec, prefix): - configure() - make() - make('install') diff --git a/lib/spack/docs/tutorial/examples/4.package.py b/lib/spack/docs/tutorial/examples/4.package.py deleted file mode 100644 index 626e7d3fec..0000000000 --- a/lib/spack/docs/tutorial/examples/4.package.py +++ /dev/null @@ -1,27 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Mpileaks(Package): - """Tool to detect and report MPI objects like MPI_Requests and - MPI_Datatypes.""" - - homepage = "https://github.com/LLNL/mpileaks" - url = "https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" - - version('1.0', sha256='2e34cc4505556d1c1f085758e26f2f8eea0972db9382f051b2dcfb1d7d9e1825') - - depends_on('mpi') - depends_on('adept-utils') - depends_on('callpath') - - def install(self, spec, prefix): - configure('--prefix={0}'.format(prefix), - '--with-adept-utils={0}'.format(spec['adept-utils'].prefix), - '--with-callpath={0}'.format(spec['callpath'].prefix)) - make() - make('install') diff --git a/lib/spack/docs/tutorial/examples/5.package.py b/lib/spack/docs/tutorial/examples/5.package.py deleted file mode 100644 index 9a1dec2079..0000000000 --- a/lib/spack/docs/tutorial/examples/5.package.py +++ /dev/null @@ -1,42 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Mpileaks(Package): - """Tool to detect and report MPI objects like MPI_Requests and - MPI_Datatypes.""" - - homepage = "https://github.com/LLNL/mpileaks" - url = "https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" - - version('1.0', sha256='2e34cc4505556d1c1f085758e26f2f8eea0972db9382f051b2dcfb1d7d9e1825') - - variant('stackstart', values=int, default=0, - description='Specify the number of stack frames to truncate') - - depends_on('mpi') - depends_on('adept-utils') - depends_on('callpath') - - def install(self, spec, prefix): - stackstart = int(spec.variants['stackstart'].value) - - args = [ - '--prefix={0}'.format(prefix), - '--with-adept-utils={0}'.format(spec['adept-utils'].prefix), - '--with-callpath={0}'.format(spec['callpath'].prefix), - ] - - if stackstart: - args.extend([ - '--with-stack-start-c={0}'.format(stackstart), - '--with-stack-start-fortran={0}'.format(stackstart) - ]) - - configure(*args) - make() - make('install') diff --git a/lib/spack/docs/tutorial/examples/Autotools/0.package.py b/lib/spack/docs/tutorial/examples/Autotools/0.package.py deleted file mode 100644 index 213965e5c6..0000000000 --- a/lib/spack/docs/tutorial/examples/Autotools/0.package.py +++ /dev/null @@ -1,27 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Mpileaks(AutotoolsPackage): - """Tool to detect and report leaked MPI objects like MPI_Requests and - MPI_Datatypes.""" - - homepage = "https://github.com/hpc/mpileaks" - url = "https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" - - version('1.0', '8838c574b39202a57d7c2d68692718aa') - - depends_on("mpi") - depends_on("adept-utils") - depends_on("callpath") - - def install(self, spec, prefix): - configure("--prefix=" + prefix, - "--with-adept-utils=" + spec['adept-utils'].prefix, - "--with-callpath=" + spec['callpath'].prefix) - make() - make("install") diff --git a/lib/spack/docs/tutorial/examples/Autotools/1.package.py b/lib/spack/docs/tutorial/examples/Autotools/1.package.py deleted file mode 100644 index 639f130596..0000000000 --- a/lib/spack/docs/tutorial/examples/Autotools/1.package.py +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Mpileaks(AutotoolsPackage): - """Tool to detect and report leaked MPI objects like MPI_Requests and - MPI_Datatypes.""" - - homepage = "https://github.com/hpc/mpileaks" - url = "https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" - - version('1.0', '8838c574b39202a57d7c2d68692718aa') - - variant("stackstart", values=int, default=0, - description="Specify the number of stack frames to truncate") - - depends_on("mpi") - depends_on("adept-utils") - depends_on("callpath") - - def configure_args(self): - stackstart = int(self.spec.variants['stackstart'].value) - args = ["--with-adept-utils=" + spec['adept-utils'].prefix, - "--with-callpath=" + spec['callpath'].prefix] - if stackstart: - args.extend(['--with-stack-start-c=%s' % stackstart, - '--with-stack-start-fortran=%s' % stackstart]) - return args diff --git a/lib/spack/docs/tutorial/examples/Cmake/0.package.py b/lib/spack/docs/tutorial/examples/Cmake/0.package.py deleted file mode 100644 index b8c96a1929..0000000000 --- a/lib/spack/docs/tutorial/examples/Cmake/0.package.py +++ /dev/null @@ -1,41 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -# -# This is a template package file for Spack. We've put "FIXME" -# next to all the things you'll want to change. Once you've handled -# them, you can save this file and test your package like this: -# -# spack install callpath -# -# You can edit this file again by typing: -# -# spack edit callpath -# -# See the Spack documentation for more information on packaging. -# If you submit this package back to Spack as a pull request, -# please first remove this boilerplate and all FIXME comments. -# -from spack import * - - -class Callpath(CMakePackage): - """FIXME: Put a proper description of your package here.""" - - # FIXME: Add a proper url for your package's homepage here. - homepage = "http://www.example.com" - url = "https://github.com/llnl/callpath/archive/v1.0.1.tar.gz" - - version('1.0.3', 'c89089b3f1c1ba47b09b8508a574294a') - - # FIXME: Add dependencies if required. - # depends_on('foo') - - def cmake_args(self): - # FIXME: Add arguments other than - # FIXME: CMAKE_INSTALL_PREFIX and CMAKE_BUILD_TYPE - # FIXME: If not needed delete this function - args = [] - return args diff --git a/lib/spack/docs/tutorial/examples/Cmake/1.package.py b/lib/spack/docs/tutorial/examples/Cmake/1.package.py deleted file mode 100644 index 8a0d9d0e82..0000000000 --- a/lib/spack/docs/tutorial/examples/Cmake/1.package.py +++ /dev/null @@ -1,23 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Callpath(CMakePackage): - """Library for representing callpaths consistently in - distributed-memory performance tools.""" - - homepage = "https://github.com/llnl/callpath" - url = "https://github.com/llnl/callpath/archive/v1.0.3.tar.gz" - - version('1.0.3', 'c89089b3f1c1ba47b09b8508a574294a') - - depends_on("elf", type="link") - depends_on("libdwarf") - depends_on("dyninst") - depends_on("adept-utils") - depends_on("mpi") - depends_on("cmake@2.8:", type="build") diff --git a/lib/spack/docs/tutorial/examples/Cmake/2.package.py b/lib/spack/docs/tutorial/examples/Cmake/2.package.py deleted file mode 100644 index 2b4faca90d..0000000000 --- a/lib/spack/docs/tutorial/examples/Cmake/2.package.py +++ /dev/null @@ -1,33 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Callpath(CMakePackage): - """Library for representing callpaths consistently in - distributed-memory performance tools.""" - - homepage = "https://github.com/llnl/callpath" - url = "https://github.com/llnl/callpath/archive/v1.0.3.tar.gz" - - version('1.0.3', 'c89089b3f1c1ba47b09b8508a574294a') - - depends_on("elf", type="link") - depends_on("libdwarf") - depends_on("dyninst") - depends_on("adept-utils") - depends_on("mpi") - depends_on("cmake@2.8:", type="build") - - def cmake_args(self): - args = ["-DCALLPATH_WALKER=dyninst"] - - if self.spec.satisfies("^dyninst@9.3.0:"): - std.flag = self.compiler.cxx_flag - args.append("-DCMAKE_CXX_FLAGS='{0}' -fpermissive'".format( - std_flag)) - - return args diff --git a/lib/spack/docs/tutorial/examples/Makefile/0.package.py b/lib/spack/docs/tutorial/examples/Makefile/0.package.py deleted file mode 100644 index 8ad252f5b3..0000000000 --- a/lib/spack/docs/tutorial/examples/Makefile/0.package.py +++ /dev/null @@ -1,26 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Bowtie(MakefilePackage): - """FIXME: Put a proper description of your package here.""" - - # FIXME: Add a proper url for your package's homepage here. - homepage = "http://www.example.com" - url = "https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip" - - version('1.2.1.1', 'ec06265730c5f587cd58bcfef6697ddf') - - # FIXME: Add dependencies if required. - # depends_on('foo') - - def edit(self, spec, prefix): - # FIXME: Edit the Makefile if necessary - # FIXME: If not needed delete this function - # makefile = FileFilter('Makefile') - # makefile.filter('CC = .*', 'CC = cc') - return diff --git a/lib/spack/docs/tutorial/examples/Makefile/1.package.py b/lib/spack/docs/tutorial/examples/Makefile/1.package.py deleted file mode 100644 index 16988dc1b0..0000000000 --- a/lib/spack/docs/tutorial/examples/Makefile/1.package.py +++ /dev/null @@ -1,27 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Bowtie(MakefilePackage): - """Bowtie is an ultrafast, memory efficient short read aligner - for short DNA sequences (reads) from next-gen sequencers.""" - - homepage = "https://sourceforge.net/projects/bowtie-bio/" - url = "https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip" - - version('1.2.1.1', 'ec06265730c5f587cd58bcfef6697ddf') - - variant("tbb", default=False, description="Use Intel thread building block") - - depends_on("tbb", when="+tbb") - - def edit(self, spec, prefix): - # FIXME: Edit the Makefile if necessary - # FIXME: If not needed delete this function - # makefile = FileFilter('Makefile') - # makefile.filter('CC = .*', 'CC = cc') - return diff --git a/lib/spack/docs/tutorial/examples/Makefile/2.package.py b/lib/spack/docs/tutorial/examples/Makefile/2.package.py deleted file mode 100644 index 0973b2028e..0000000000 --- a/lib/spack/docs/tutorial/examples/Makefile/2.package.py +++ /dev/null @@ -1,25 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Bowtie(MakefilePackage): - """Bowtie is an ultrafast, memory efficient short read aligner - for short DNA sequences (reads) from next-gen sequencers.""" - - homepage = "https://sourceforge.net/projects/bowtie-bio/" - url = "https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip" - - version('1.2.1.1', 'ec06265730c5f587cd58bcfef6697ddf') - - variant("tbb", default=False, description="Use Intel thread building block") - - depends_on("tbb", when="+tbb") - - def edit(self, spec, prefix): - makefile = FileFilter("Makefile") - makefile.filter('CC= .*', 'CC = ' + env['CC']) - makefile.filter('CXX = .*', 'CXX = ' + env['CXX']) diff --git a/lib/spack/docs/tutorial/examples/Makefile/3.package.py b/lib/spack/docs/tutorial/examples/Makefile/3.package.py deleted file mode 100644 index 6a1942e627..0000000000 --- a/lib/spack/docs/tutorial/examples/Makefile/3.package.py +++ /dev/null @@ -1,36 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class Bowtie(MakefilePackage): - """Bowtie is an ultrafast, memory efficient short read aligner - for short DNA sequences (reads) from next-gen sequencers.""" - - homepage = "https://sourceforge.net/projects/bowtie-bio/" - url = "https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip" - - version('1.2.1.1', 'ec06265730c5f587cd58bcfef6697ddf') - - variant("tbb", default=False, description="Use Intel thread building block") - - depends_on("tbb", when="+tbb") - - def edit(self, spec, prefix): - makefile = FileFilter("Makefile") - makefile.filter('CC= .*', 'CC = ' + env['CC']) - makefile.filter('CXX = .*', 'CXX = ' + env['CXX']) - - @property - def build_targets(self): - if "+tbb" in spec: - return [] - else: - return ["NO_TBB=1"] - - @property - def install_targets(self): - return ['prefix={0}'.format(self.prefix), 'install'] diff --git a/lib/spack/docs/tutorial/examples/PyPackage/0.package.py b/lib/spack/docs/tutorial/examples/PyPackage/0.package.py deleted file mode 100644 index 9ad94189d9..0000000000 --- a/lib/spack/docs/tutorial/examples/PyPackage/0.package.py +++ /dev/null @@ -1,41 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -# -# This is a template package file for Spack. We've put "FIXME" -# next to all the things you'll want to change. Once you've handled -# them, you can save this file and test your package like this: -# -# spack install py-pandas -# -# You can edit this file again by typing: -# -# spack edit py-pandas -# -# See the Spack documentation for more information on packaging. -# If you submit this package back to Spack as a pull request, -# please first remove this boilerplate and all FIXME comments. -# -from spack import * - - -class PyPandas(PythonPackage): - """FIXME: Put a proper description of your package here.""" - - # FIXME: Add a proper url for your package's homepage here. - homepage = "http://www.example.com" - url = "https://pypi.io/packages/source/p/pandas/pandas-0.19.0.tar.gz" - - version('0.19.0', 'bc9bb7188e510b5d44fbdd249698a2c3') - - # FIXME: Add dependencies if required. - # depends_on('py-setuptools', type='build') - # depends_on('py-foo', type=('build', 'run')) - - def build_args(self, spec, prefix): - # FIXME: Add arguments other than --prefix - # FIXME: If not needed delete this function - args = [] - return args diff --git a/lib/spack/docs/tutorial/examples/PyPackage/1.package.py b/lib/spack/docs/tutorial/examples/PyPackage/1.package.py deleted file mode 100644 index 8ac335d4a6..0000000000 --- a/lib/spack/docs/tutorial/examples/PyPackage/1.package.py +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other -# Spack Project Developers. See the top-level COPYRIGHT file for details. -# -# SPDX-License-Identifier: (Apache-2.0 OR MIT) - -from spack import * - - -class PyPandas(PythonPackage): - """pandas is a Python package providing fast, flexible, and expressive - data structures designed to make working with relational or - labeled data both easy and intuitive. It aims to be the - fundamental high-level building block for doing practical, real - world data analysis in Python. Additionally, it has the broader - goal of becoming the most powerful and flexible open source data - analysis / manipulation tool available in any language. - """ - homepage = "http://pandas.pydata.org/" - url = "https://pypi.io/packages/source/p/pandas/pandas-0.19.0.tar.gz" - - version('0.19.0', 'bc9bb7188e510b5d44fbdd249698a2c3') - version('0.18.0', 'f143762cd7a59815e348adf4308d2cf6') - version('0.16.1', 'fac4f25748f9610a3e00e765474bdea8') - version('0.16.0', 'bfe311f05dc0c351f8955fbd1e296e73') - - depends_on('py-dateutil', type=('build', 'run')) - depends_on('py-numpy', type=('build', 'run')) - depends_on('py-setuptools', type='build') - depends_on('py-cython', type='build') - depends_on('py-pytz', type=('build', 'run')) - depends_on('py-numexpr', type=('build', 'run')) - depends_on('py-bottleneck', type=('build', 'run')) diff --git a/lib/spack/docs/tutorial/sc16-tutorial-slide-preview.png b/lib/spack/docs/tutorial/sc16-tutorial-slide-preview.png Binary files differdeleted file mode 100644 index e7f9b2e323..0000000000 --- a/lib/spack/docs/tutorial/sc16-tutorial-slide-preview.png +++ /dev/null diff --git a/lib/spack/docs/tutorial_advanced_packaging.rst b/lib/spack/docs/tutorial_advanced_packaging.rst deleted file mode 100644 index 876a6e8f70..0000000000 --- a/lib/spack/docs/tutorial_advanced_packaging.rst +++ /dev/null @@ -1,515 +0,0 @@ -.. Copyright 2013-2019 Lawrence Livermore National Security, LLC and other - Spack Project Developers. See the top-level COPYRIGHT file for details. - - SPDX-License-Identifier: (Apache-2.0 OR MIT) - -.. _advanced-packaging-tutorial: - -============================ -Advanced Topics in Packaging -============================ - -Spack tries to automatically configure packages with information from -dependencies such that all you need to do is to list the dependencies -(i.e., with the ``depends_on`` directive) and the build system (for example -by deriving from :code:`CmakePackage`). - -However, there are many special cases. Often you need to retrieve details -about dependencies to set package-specific configuration options, or to -define package-specific environment variables used by the package's build -system. This tutorial covers how to retrieve build information from -dependencies, and how you can automatically provide important information to -dependents in your package. - ----------------------- -Setup for the tutorial ----------------------- - -.. note:: - - We do not recommend doing this section of the tutorial in a - production Spack instance. - -The tutorial uses custom package definitions with missing sections that -will be filled in during the tutorial. These package definitions are stored -in a separate package repository, which can be enabled with: - -.. code-block:: console - - $ spack repo add --scope=site var/spack/repos/tutorial - -This section of the tutorial may also require a newer version of -gcc. If you have not already installed gcc@7.2.0 and added it to your -configuration, you can do so with: - -.. code-block:: console - - $ spack install gcc@7.2.0 %gcc@5.4.0 - $ spack compiler add --scope=site `spack location -i gcc@7.2.0 %gcc@5.4.0` - -If you are using the tutorial docker image, all dependency packages -will have been installed. Otherwise, to install these packages you can use -the following commands: - -.. code-block:: console - - $ spack install openblas - $ spack install netlib-lapack - $ spack install mpich - -Now, you are ready to set your preferred ``EDITOR`` and continue with -the rest of the tutorial. - -.. note:: - - Several of these packages depend on an MPI implementation. You can use - OpenMPI if you install it from scratch, but this is slow (>10 min.). - A binary cache of MPICH may be provided, in which case you can force - the package to use it and install quickly. All tutorial examples with - packages that depend on MPICH include the spec syntax for building with it - -.. _adv_pkg_tutorial_start: - ---------------------------------------- -Modifying a package's build environment ---------------------------------------- - -Spack sets up several environment variables like ``PATH`` by default to aid in -building a package, but many packages make use of environment variables which -convey specific information about their dependencies (e.g., ``MPICC``). -This section covers how to update your Spack packages so that package-specific -environment variables are defined at build-time. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Set environment variables in dependent packages at build-time -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Dependencies can set environment variables that are required when their -dependents build. For example, when a package depends on a python extension -like py-numpy, Spack's ``python`` package will add it to ``PYTHONPATH`` -so it is available at build time; this is required because the default setup -that spack does is not sufficient for python to import modules. - -Any package can override the -:py:func:`setup_dependent_build_environment <spack.package.PackageBase.setup_dependent_build_environment>` -method to setup the build environment for a dependent. -This method takes as an argument a :py:class:`EnvironmentModifications <spack.util.environment.EnvironmentModifications>` -object which includes convenience methods to update the environment. For -example, an MPI implementation can set ``MPICC`` for packages that depend on it: - -.. code-block:: python - - def setup_dependent_build_environment(self, env, dependent_spec): - env.set('MPICC', join_path(self.prefix.bin, 'mpicc')) - -In this case packages that depend on ``mpi`` will have ``MPICC`` defined in -their environment when they build. This section is focused on setting up the -build-time environment but it's worth noting that a similar method called -:py:func:`setup_dependent_run_environment <spack.package.PackageBase.setup_dependent_run_environment>` -can be used to code modifications that will be included in Spack's automatically-generated -module files. - -We can practice by editing the ``mpich`` package to set the ``MPICC`` -environment variable in the build-time environment of dependent packages. - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack edit mpich - -Once you're finished, the method should look like this: - -.. code-block:: python - - def setup_dependent_build_environment(self, env, dependent_spec): - env.set('MPICC', join_path(self.prefix.bin, 'mpicc')) - env.set('MPICXX', join_path(self.prefix.bin, 'mpic++')) - env.set('MPIF77', join_path(self.prefix.bin, 'mpif77')) - env.set('MPIF90', join_path(self.prefix.bin, 'mpif90')) - - env.set('MPICH_CC', spack_cc) - env.set('MPICH_CXX', spack_cxx) - env.set('MPICH_F77', spack_f77) - env.set('MPICH_F90', spack_fc) - env.set('MPICH_FC', spack_fc) - -At this point we can, for instance, install ``netlib-scalapack`` with -``mpich``: - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack install netlib-scalapack ^mpich - ... - ==> Created stage in /usr/local/var/spack/stage/netlib-scalapack-2.0.2-km7tsbgoyyywonyejkjoojskhc5knz3z - ==> No patches needed for netlib-scalapack - ==> Building netlib-scalapack [CMakePackage] - ==> Executing phase: 'cmake' - ==> Executing phase: 'build' - ==> Executing phase: 'install' - ==> Successfully installed netlib-scalapack - Fetch: 0.01s. Build: 3m 59.86s. Total: 3m 59.87s. - [+] /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/netlib-scalapack-2.0.2-km7tsbgoyyywonyejkjoojskhc5knz3z - - -and double check the environment logs to verify that every variable was -set to the correct value. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Set environment variables in your own package -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Packages can override the -:py:func:`setup_build_environment <spack.package.PackageBase.setup_build_environment>` -or the -:py:func:`setup_run_environment <spack.package.PackageBase.setup_run_environment>` -methods to modify their own build-time or run-time environment respectively. -An example of a package that overrides both methods is ``qt``: - -.. code-block:: python - - def setup_build_environment(self, env): - env.set('MAKEFLAGS', '-j{0}'.format(make_jobs)) - - def setup_run_environment(self, env): - env.set('QTDIR', self.prefix) - -When ``qt`` builds, ``MAKEFLAGS`` will be defined in the environment. Likewise, when a -module file is created for ``qt`` it will contain commands to define ``QTDIR`` appropriately. - -To contrast with ``qt``'s -:py:func:`setup_dependent_build_environment <spack.package.PackageBase.setup_dependent_build_environment>` -function: - -.. code-block:: python - - def setup_dependent_build_environment(self, env, dependent_spec): - env.set('QTDIR', self.prefix) - -Let's see how it works by completing the ``elpa`` package: - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack edit elpa - -In the end your method should look like: - -.. code-block:: python - - def setup_build_environment(self, env): - spec = self.spec - - env.set('CC', spec['mpi'].mpicc) - env.set('FC', spec['mpi'].mpifc) - env.set('CXX', spec['mpi'].mpicxx) - env.set('SCALAPACK_LDFLAGS', spec['scalapack'].libs.joined()) - - env.append_flags('LDFLAGS', spec['lapack'].libs.search_flags) - env.append_flags('LIBS', spec['lapack'].libs.link_flags) - -At this point it's possible to proceed with the installation of ``elpa ^mpich`` - ------------------------------- -Retrieving library information ------------------------------- - -Although Spack attempts to help packages locate their dependency libraries -automatically (e.g. by setting ``PKG_CONFIG_PATH`` and ``CMAKE_PREFIX_PATH``), -a package may have unique configuration options that are required to locate -libraries. When a package needs information about dependency libraries, the -general approach in Spack is to query the dependencies for the locations of -their libraries and set configuration options accordingly. By default most -Spack packages know how to automatically locate their libraries. This section -covers how to retrieve library information from dependencies and how to locate -libraries when the default logic doesn't work. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Accessing dependency libraries -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If you need to access the libraries of a dependency, you can do so -via the ``libs`` property of the spec, for example in the ``arpack-ng`` -package: - -.. code-block:: python - - def install(self, spec, prefix): - lapack_libs = spec['lapack'].libs.joined(';') - blas_libs = spec['blas'].libs.joined(';') - - cmake(*[ - '-DLAPACK_LIBRARIES={0}'.format(lapack_libs), - '-DBLAS_LIBRARIES={0}'.format(blas_libs) - ], '.') - -Note that ``arpack-ng`` is querying virtual dependencies, which Spack -automatically resolves to the installed implementation (e.g. ``openblas`` -for ``blas``). - -We've started work on a package for ``armadillo``. You should open it, -read through the comment that starts with ``# TUTORIAL:`` and complete -the ``cmake_args`` section: - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack edit armadillo - -If you followed the instructions in the package, when you are finished your -``cmake_args`` method should look like: - -.. code-block:: python - - def cmake_args(self): - spec = self.spec - - return [ - # ARPACK support - '-DARPACK_LIBRARY={0}'.format(spec['arpack-ng'].libs.joined(";")), - # BLAS support - '-DBLAS_LIBRARY={0}'.format(spec['blas'].libs.joined(";")), - # LAPACK support - '-DLAPACK_LIBRARY={0}'.format(spec['lapack'].libs.joined(";")), - # SuperLU support - '-DSuperLU_INCLUDE_DIR={0}'.format(spec['superlu'].prefix.include), - '-DSuperLU_LIBRARY={0}'.format(spec['superlu'].libs.joined(";")), - # HDF5 support - '-DDETECT_HDF5={0}'.format('ON' if '+hdf5' in spec else 'OFF') - ] - -As you can see, getting the list of libraries that your dependencies provide -is as easy as accessing the their ``libs`` attribute. Furthermore, the interface -remains the same whether you are querying regular or virtual dependencies. - -At this point you can complete the installation of ``armadillo`` using ``openblas`` -as a LAPACK provider (``armadillo ^openblas ^mpich``): - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack install armadillo ^openblas ^mpich - ==> pkg-config is already installed in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkg-config-0.29.2-ae2hwm7q57byfbxtymts55xppqwk7ecj - ... - ==> superlu is already installed in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/superlu-5.2.1-q2mbtw2wo4kpzis2e2n227ip2fquxrno - ==> Installing armadillo - ==> Using cached archive: /usr/local/var/spack/cache/armadillo/armadillo-8.100.1.tar.xz - ==> Staging archive: /usr/local/var/spack/stage/armadillo-8.100.1-n2eojtazxbku6g4l5izucwwgnpwz77r4/armadillo-8.100.1.tar.xz - ==> Created stage in /usr/local/var/spack/stage/armadillo-8.100.1-n2eojtazxbku6g4l5izucwwgnpwz77r4 - ==> Applied patch undef_linux.patch - ==> Building armadillo [CMakePackage] - ==> Executing phase: 'cmake' - ==> Executing phase: 'build' - ==> Executing phase: 'install' - ==> Successfully installed armadillo - Fetch: 0.01s. Build: 3.96s. Total: 3.98s. - [+] /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/armadillo-8.100.1-n2eojtazxbku6g4l5izucwwgnpwz77r4 - -Hopefully the installation went fine and the code we added expanded to the right list -of semicolon separated libraries (you are encouraged to open ``armadillo``'s -build logs to double check). - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Providing libraries to dependents -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Spack provides a default implementation for ``libs`` which often works -out of the box. A user can write a package definition without having to -implement a ``libs`` property and dependents can retrieve its libraries -as shown in the above section. However, the default implementation assumes that -libraries follow the naming scheme ``lib<package name>.so`` (or e.g. -``lib<package name>.a`` for static libraries). Packages which don't -follow this naming scheme must implement this function themselves, e.g. -``opencv``: - -.. code-block:: python - - @property - def libs(self): - shared = "+shared" in self.spec - return find_libraries( - "libopencv_*", root=self.prefix, shared=shared, recurse=True - ) - -This issue is common for packages which implement an interface (i.e. -virtual package providers in Spack). If we try to build another version of -``armadillo`` tied to ``netlib-lapack`` (``armadillo ^netlib-lapack ^mpich``) -we'll notice that this time the installation won't complete: - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack install armadillo ^netlib-lapack ^mpich - ==> pkg-config is already installed in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkg-config-0.29.2-ae2hwm7q57byfbxtymts55xppqwk7ecj - ... - ==> openmpi is already installed in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f - ==> Installing arpack-ng - ==> Using cached archive: /usr/local/var/spack/cache/arpack-ng/arpack-ng-3.5.0.tar.gz - ==> Already staged arpack-ng-3.5.0-bloz7cqirpdxj33pg7uj32zs5likz2un in /usr/local/var/spack/stage/arpack-ng-3.5.0-bloz7cqirpdxj33pg7uj32zs5likz2un - ==> No patches needed for arpack-ng - ==> Building arpack-ng [Package] - ==> Executing phase: 'install' - ==> Error: RuntimeError: Unable to recursively locate netlib-lapack libraries in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/netlib-lapack-3.6.1-jjfe23wgt7nkjnp2adeklhseg3ftpx6z - RuntimeError: RuntimeError: Unable to recursively locate netlib-lapack libraries in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/netlib-lapack-3.6.1-jjfe23wgt7nkjnp2adeklhseg3ftpx6z - - /usr/local/var/spack/repos/builtin/packages/arpack-ng/package.py:105, in install: - 5 options.append('-DCMAKE_INSTALL_NAME_DIR:PATH=%s/lib' % prefix) - 6 - 7 # Make sure we use Spack's blas/lapack: - >> 8 lapack_libs = spec['lapack'].libs.joined(';') - 9 blas_libs = spec['blas'].libs.joined(';') - 10 - 11 options.extend([ - - See build log for details: - /usr/local/var/spack/stage/arpack-ng-3.5.0-bloz7cqirpdxj33pg7uj32zs5likz2un/arpack-ng-3.5.0/spack-build-out.txt - -Unlike ``openblas`` which provides a library named ``libopenblas.so``, -``netlib-lapack`` provides ``liblapack.so``, so it needs to implement -customized library search logic. Let's edit it: - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack edit netlib-lapack - -and follow the instructions in the ``# TUTORIAL:`` comment as before. -What we need to implement is: - -.. code-block:: python - - @property - def lapack_libs(self): - shared = True if '+shared' in self.spec else False - return find_libraries( - 'liblapack', root=self.prefix, shared=shared, recursive=True - ) - -i.e., a property that returns the correct list of libraries for the LAPACK interface. - -We use the name ``lapack_libs`` rather than ``libs`` because -``netlib-lapack`` can also provide ``blas``, and when it does it is provided -as a separate library file. Using this name ensures that when -dependents ask for ``lapack`` libraries, ``netlib-lapack`` will retrieve only -the libraries associated with the ``lapack`` interface. Now we can finally -install ``armadillo ^netlib-lapack ^mpich``: - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack install armadillo ^netlib-lapack ^mpich - ... - - ==> Building armadillo [CMakePackage] - ==> Executing phase: 'cmake' - ==> Executing phase: 'build' - ==> Executing phase: 'install' - ==> Successfully installed armadillo - Fetch: 0.01s. Build: 3.75s. Total: 3.76s. - [+] /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/armadillo-8.100.1-sxmpu5an4dshnhickh6ykchyfda7jpyn - -Since each implementation of a virtual package is responsible for locating the -libraries associated with the interfaces it provides, dependents do not need -to include special-case logic for different implementations and for example -need only ask for :code:`spec['blas'].libs`. - ----------------------- -Other Packaging Topics ----------------------- - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Attach attributes to other packages -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Build tools usually also provide a set of executables that can be used -when another package is being installed. Spack gives you the opportunity -to monkey-patch dependent modules and attach attributes to them. This -helps make the packager experience as similar as possible to what would -have been the manual installation of the same package. - -An example here is the ``automake`` package, which overrides -:py:func:`setup_dependent_package <spack.package.PackageBase.setup_dependent_package>`: - -.. code-block:: python - - def setup_dependent_package(self, module, dependent_spec): - # Automake is very likely to be a build dependency, - # so we add the tools it provides to the dependent module - executables = ['aclocal', 'automake'] - for name in executables: - setattr(module, name, self._make_executable(name)) - -so that every other package that depends on it can use directly ``aclocal`` -and ``automake`` with the usual function call syntax of :py:class:`Executable <spack.util.executable.Executable>`: - -.. code-block:: python - - aclocal('--force') - -^^^^^^^^^^^^^^^^^^^^^^^ -Extra query parameters -^^^^^^^^^^^^^^^^^^^^^^^ - -An advanced feature of the Spec's build-interface protocol is the support -for extra parameters after the subscript key. In fact, any of the keys used in the query -can be followed by a comma-separated list of extra parameters which can be -inspected by the package receiving the request to fine-tune a response. - -Let's look at an example and try to install ``netcdf ^mpich``: - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack install netcdf ^mpich - ==> libsigsegv is already installed in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libsigsegv-2.11-fypapcprssrj3nstp6njprskeyynsgaz - ==> m4 is already installed in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/m4-1.4.18-r5envx3kqctwwflhd4qax4ahqtt6x43a - ... - ==> Error: AttributeError: 'list' object has no attribute 'search_flags' - AttributeError: AttributeError: 'list' object has no attribute 'search_flags' - - /usr/local/var/spack/repos/builtin/packages/netcdf/package.py:207, in configure_args: - 50 # used instead. - 51 hdf5_hl = self.spec['hdf5:hl'] - 52 CPPFLAGS.append(hdf5_hl.headers.cpp_flags) - >> 53 LDFLAGS.append(hdf5_hl.libs.search_flags) - 54 - 55 if '+parallel-netcdf' in self.spec: - 56 config_args.append('--enable-pnetcdf') - - See build log for details: - /usr/local/var/spack/stage/netcdf-4.4.1.1-gk2xxhbqijnrdwicawawcll4t3c7dvoj/netcdf-4.4.1.1/spack-build-out.txt - -We can see from the error that ``netcdf`` needs to know how to link the *high-level interface* -of ``hdf5``, and thus passes the extra parameter ``hl`` after the request to retrieve it. -Clearly the implementation in the ``hdf5`` package is not complete, and we need to fix it: - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack edit hdf5 - -If you followed the instructions correctly, the code added to the -``lib`` property should be similar to: - -.. code-block:: python - :emphasize-lines: 1 - - query_parameters = self.spec.last_query.extra_parameters - key = tuple(sorted(query_parameters)) - libraries = query2libraries[key] - shared = '+shared' in self.spec - return find_libraries( - libraries, root=self.prefix, shared=shared, recurse=True - ) - -where we highlighted the line retrieving the extra parameters. Now we can successfully -complete the installation of ``netcdf ^mpich``: - -.. code-block:: console - - root@advanced-packaging-tutorial:/# spack install netcdf ^mpich - ==> libsigsegv is already installed in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libsigsegv-2.11-fypapcprssrj3nstp6njprskeyynsgaz - ==> m4 is already installed in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/m4-1.4.18-r5envx3kqctwwflhd4qax4ahqtt6x43a - ... - ==> Installing netcdf - ==> Using cached archive: /usr/local/var/spack/cache/netcdf/netcdf-4.4.1.1.tar.gz - ==> Already staged netcdf-4.4.1.1-gk2xxhbqijnrdwicawawcll4t3c7dvoj in /usr/local/var/spack/stage/netcdf-4.4.1.1-gk2xxhbqijnrdwicawawcll4t3c7dvoj - ==> Already patched netcdf - ==> Building netcdf [AutotoolsPackage] - ==> Executing phase: 'autoreconf' - ==> Executing phase: 'configure' - ==> Executing phase: 'build' - ==> Executing phase: 'install' - ==> Successfully installed netcdf - Fetch: 0.01s. Build: 24.61s. Total: 24.62s. - [+] /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/netcdf-4.4.1.1-gk2xxhbqijnrdwicawawcll4t3c7dvoj diff --git a/lib/spack/docs/tutorial_basics.rst b/lib/spack/docs/tutorial_basics.rst deleted file mode 100644 index 9a6a2074b3..0000000000 --- a/lib/spack/docs/tutorial_basics.rst +++ /dev/null @@ -1,1736 +0,0 @@ -.. Copyright 2013-2019 Lawrence Livermore National Security, LLC and other - Spack Project Developers. See the top-level COPYRIGHT file for details. - - SPDX-License-Identifier: (Apache-2.0 OR MIT) - -.. _basics-tutorial: - -========================================= -Basic Installation Tutorial -========================================= - -This tutorial will guide you through the process of installing -software using Spack. We will first cover the `spack install` command, -focusing on the power of the spec syntax and the flexibility it gives -to users. We will also cover the `spack find` command for viewing -installed packages and the `spack uninstall` command. Finally, we will -touch on how Spack manages compilers, especially as it relates to -using Spack-built compilers within Spack. We will include full output -from all of the commands demonstrated, although we will frequently -call attention to only small portions of that output (or merely to the -fact that it succeeded). The provided output is all from an AWS -instance running Ubuntu 16.04 - -.. _basics-tutorial-install: - ----------------- -Installing Spack ----------------- - -Spack works out of the box. Simply clone spack and get going. We will -clone Spack and immediately checkout the most recent release, v0.12. - -.. code-block:: console - - $ git clone https://github.com/spack/spack - git clone https://github.com/spack/spack - Cloning into 'spack'... - remote: Enumerating objects: 68, done. - remote: Counting objects: 100% (68/68), done. - remote: Compressing objects: 100% (56/56), done. - remote: Total 135389 (delta 40), reused 16 (delta 9), pack-reused 135321 - Receiving objects: 100% (135389/135389), 47.31 MiB | 1.01 MiB/s, done. - Resolving deltas: 100% (64414/64414), done. - Checking connectivity... done. - $ cd spack - $ git checkout releases/v0.12 - Branch releases/v0.12 set up to track remote branch releases/v0.12 from origin. - Switched to a new branch 'releases/v0.12' - -Next add Spack to your path. Spack has some nice command line -integration tools, so instead of simply appending to your ``PATH`` -variable, source the spack setup script. Then add Spack to your path. - -.. code-block:: console - - $ . share/spack/setup-env.sh - -You're good to go! - ------------------ -What is in Spack? ------------------ - -The ``spack list`` command shows available packages. - -.. code-block:: console - - $ spack list - ==> 2907 packages. - abinit libgpuarray py-espresso r-mlrmbo - abyss libgridxc py-espressopp r-mmwrweek - accfft libgtextutils py-et-xmlfile r-mnormt - ... - -The ``spack list`` command can also take a query string. Spack -automatically adds wildcards to both ends of the string. For example, -we can view all available python packages. - -.. code-block:: console - - $ spack list py- - ==> 479 packages. - lumpy-sv py-funcsigs py-numpydoc py-utililib - perl-file-copy-recursive py-functools32 py-olefile py-pywavelets - py-3to2 py-future py-ont-fast5-api py-pyyaml - ... - -------------------- -Installing Packages -------------------- - -Installing a package with Spack is very simple. To install a piece of -software, simply type ``spack install <package_name>``. - -.. code-block:: console - - $ spack install zlib - ==> Installing zlib - ==> Searching for binary cache of zlib - ==> Warning: No Spack mirrors are currently configured - ==> No binary for zlib found: installing from source - ==> Fetching http://zlib.net/fossils/zlib-1.2.11.tar.gz - ######################################################################## 100.0% - ==> Staging archive: /home/spack1/spack/var/spack/stage/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb/zlib-1.2.11.tar.gz - ==> Created stage in /home/spack1/spack/var/spack/stage/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - ==> No patches needed for zlib - ==> Building zlib [Package] - ==> Executing phase: 'install' - ==> Successfully installed zlib - Fetch: 3.27s. Build: 2.18s. Total: 5.44s. - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - -Spack can install software either from source or from a binary -cache. Packages in the binary cache are signed with GPG for -security. For the tutorial we have prepared a binary cache so you -don't have to wait on slow compilation from source. To be able to -install from the binary cache, we will need to configure Spack with -the location of the binary cache and trust the GPG key that the binary -cache was signed with. - -.. code-block:: console - - $ spack mirror add tutorial /mirror - $ spack gpg trust /mirror/public.key - gpg: keybox '/home/spack1/spack/opt/spack/gpg/pubring.kbx' created - gpg: /home/spack1/spack/opt/spack/gpg/trustdb.gpg: trustdb created - gpg: key 3B7C69B2: public key "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" imported - gpg: Total number processed: 1 - gpg: imported: 1 - -You'll learn more about configuring Spack later in the tutorial, but -for now you will be able to install the rest of the packages in the -tutorial from a binary cache using the same ``spack install`` -command. By default this will install the binary cached version if it -exists and fall back on installing from source. - -Spack's spec syntax is the interface by which we can request specific -configurations of the package. The ``%`` sigil is used to specify -compilers. - -.. code-block:: console - - $ spack install zlib %clang - ==> Installing zlib - ==> Searching for binary cache of zlib - ==> Finding buildcaches in /mirror/build_cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64-gcc-7.2.0-texinfo-6.5-cuqnfgfhhmudqp5f7upmld6ax7pratzw.spec.yaml - ######################################################################## 100.0% - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64-gcc-4.7-zlib-1.2.11-bq2wtdxakpjytk2tjr7qu23i4py2fi2r.spec.yaml - ######################################################################## 100.0% - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64-gcc-5.4.0-dyninst-9.3.2-bu6s2jzievsjkwtcnrtimc5b625j5omf.spec.yaml - ######################################################################## 100.0% - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64-gcc-7.2.0-openmpi-3.1.3-do5xfer2whhk7gc26atgs3ozr3ljbvs4.spec.yaml - ... - ==> Installing zlib from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/clang-3.8.0-2ubuntu4/zlib-1.2.11/linux-ubuntu16.04-x86_64-clang-3.8.0-2ubuntu4-zlib-1.2.11-4pt75q7qq6lygf3hgnona4lyc2uwedul.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:08:01 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed zlib from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/clang-3.8.0-2ubuntu4/zlib-1.2.11-4pt75q7qq6lygf3hgnona4lyc2uwedul - -Note that this installation is located separately from the previous -one. We will discuss this in more detail later, but this is part of what -allows Spack to support arbitrarily versioned software. - -You can check for particular versions before requesting them. We will -use the ``spack versions`` command to see the available versions, and then -install a different version of ``zlib``. - -.. code-block:: console - - $ spack versions zlib - ==> Safe versions (already checksummed): - 1.2.11 1.2.8 1.2.3 - ==> Remote versions (not yet checksummed): - 1.2.10 1.2.7 1.2.5.1 1.2.4.2 1.2.3.7 - ... - -The ``@`` sigil is used to specify versions, both of packages and of -compilers. - -.. code-block:: console - - $ spack install zlib@1.2.8 - ==> Installing zlib - ==> Searching for binary cache of zlib - ==> Finding buildcaches in /mirror/build_cache - ==> Installing zlib from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.8/linux-ubuntu16.04-x86_64-gcc-5.4.0-zlib-1.2.8-bkyl5bhuep6fmhuxzkmhqy25qefjcvzc.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:18:30 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed zlib from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.8-bkyl5bhuep6fmhuxzkmhqy25qefjcvzc - - $ spack install zlib %gcc@4.7 - ==> Installing zlib - ==> Searching for binary cache of zlib - ==> Finding buildcaches in /mirror/build_cache - ==> Installing zlib from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-4.7/zlib-1.2.11/linux-ubuntu16.04-x86_64-gcc-4.7-zlib-1.2.11-bq2wtdxakpjytk2tjr7qu23i4py2fi2r.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 04:55:30 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed zlib from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-4.7/zlib-1.2.11-bq2wtdxakpjytk2tjr7qu23i4py2fi2r - -The spec syntax also includes compiler flags. Spack accepts -``cppflags``, ``cflags``, ``cxxflags``, ``fflags``, ``ldflags``, and -``ldlibs`` parameters. The values of these fields must be quoted on -the command line if they include spaces. These values are injected -into the compile line automatically by the Spack compiler wrappers. - -.. code-block:: console - - $ spack install zlib @1.2.8 cppflags=-O3 - ==> Installing zlib - ==> Searching for binary cache of zlib - ==> Finding buildcaches in /mirror/build_cache - ==> Installing zlib from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.8/linux-ubuntu16.04-x86_64-gcc-5.4.0-zlib-1.2.8-64mns5mvdacqvlashkf7v6lqrxixhmxu.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:31:54 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed zlib from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.8-64mns5mvdacqvlashkf7v6lqrxixhmxu - -The ``spack find`` command is used to query installed packages. Note that -some packages appear identical with the default output. The ``-l`` flag -shows the hash of each package, and the ``-f`` flag shows any non-empty -compiler flags of those packages. - -.. code-block:: console - - $ spack find - ==> 5 installed packages. - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - zlib@1.2.11 - - -- linux-ubuntu16.04-x86_64 / gcc@4.7 --------------------------- - zlib@1.2.11 - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - zlib@1.2.8 zlib@1.2.8 zlib@1.2.11 - - - $ spack find -lf - ==> 5 installed packages. - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - 4pt75q7 zlib@1.2.11%clang - - - -- linux-ubuntu16.04-x86_64 / gcc@4.7 --------------------------- - bq2wtdx zlib@1.2.11%gcc - - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - bkyl5bh zlib@1.2.8%gcc - - 64mns5m zlib@1.2.8%gcc cppflags="-O3" - - 5nus6kn zlib@1.2.11%gcc - -Spack generates a hash for each spec. This hash is a function of the full -provenance of the package, so any change to the spec affects the -hash. Spack uses this value to compare specs and to generate unique -installation directories for every combinatorial version. As we move into -more complicated packages with software dependencies, we can see that -Spack reuses existing packages to satisfy a dependency only when the -existing package's hash matches the desired spec. - -.. code-block:: console - - $ spack install tcl - ==> zlib is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - ==> Installing tcl - ==> Searching for binary cache of tcl - ==> Finding buildcaches in /mirror/build_cache - ==> Installing tcl from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/tcl-8.6.8/linux-ubuntu16.04-x86_64-gcc-5.4.0-tcl-8.6.8-qhwyccywhx2i6s7ob2gvjrjtj3rnfuqt.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:07:15 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed tcl from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/tcl-8.6.8-qhwyccywhx2i6s7ob2gvjrjtj3rnfuqt - -Dependencies can be explicitly requested using the ``^`` sigil. Note that -the spec syntax is recursive. Anything we could specify about the -top-level package, we can also specify about a dependency using ``^``. - -.. code-block:: console - - $ spack install tcl ^zlib @1.2.8 %clang - ==> Installing zlib - ==> Searching for binary cache of zlib - ==> Finding buildcaches in /mirror/build_cache - ==> Installing zlib from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/clang-3.8.0-2ubuntu4/zlib-1.2.8/linux-ubuntu16.04-x86_64-clang-3.8.0-2ubuntu4-zlib-1.2.8-i426yu3o6lyau5fv5ljwsajfkqxj5rl5.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:09:01 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed zlib from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/clang-3.8.0-2ubuntu4/zlib-1.2.8-i426yu3o6lyau5fv5ljwsajfkqxj5rl5 - ==> Installing tcl - ==> Searching for binary cache of tcl - ==> Installing tcl from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/clang-3.8.0-2ubuntu4/tcl-8.6.8/linux-ubuntu16.04-x86_64-clang-3.8.0-2ubuntu4-tcl-8.6.8-6wc66etr7y6hgibp2derrdkf763exwvc.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:10:21 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed tcl from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/clang-3.8.0-2ubuntu4/tcl-8.6.8-6wc66etr7y6hgibp2derrdkf763exwvc - -Packages can also be referred to from the command line by their package -hash. Using the ``spack find -lf`` command earlier we saw that the hash -of our optimized installation of zlib (``cppflags="-O3"``) began with -``64mns5m``. We can now explicitly build with that package without typing -the entire spec, by using the ``/`` sigil to refer to it by hash. As with -other tools like git, you do not need to specify an *entire* hash on the -command line. You can specify just enough digits to identify a hash -uniquely. If a hash prefix is ambiguous (i.e., two or more installed -packages share the prefix) then spack will report an error. - -.. code-block:: console - - $ spack install tcl ^/64mn - ==> zlib is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.8-64mns5mvdacqvlashkf7v6lqrxixhmxu - ==> Installing tcl - ==> Searching for binary cache of tcl - ==> Finding buildcaches in /mirror/build_cache - ==> Installing tcl from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/tcl-8.6.8/linux-ubuntu16.04-x86_64-gcc-5.4.0-tcl-8.6.8-am4pbatrtga3etyusg2akmsvrswwxno2.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:11:53 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed tcl from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/tcl-8.6.8-am4pbatrtga3etyusg2akmsvrswwxno2 - -The ``spack find`` command can also take a ``-d`` flag, which can show -dependency information. Note that each package has a top-level entry, -even if it also appears as a dependency. - -.. code-block:: console - - $ spack find -ldf - ==> 9 installed packages - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - 6wc66et tcl@8.6.8%clang - i426yu3 ^zlib@1.2.8%clang - - i426yu3 zlib@1.2.8%clang - - 4pt75q7 zlib@1.2.11%clang - - - -- linux-ubuntu16.04-x86_64 / gcc@4.7 --------------------------- - bq2wtdx zlib@1.2.11%gcc - - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - am4pbat tcl@8.6.8%gcc - 64mns5m ^zlib@1.2.8%gcc cppflags="-O3" - - qhwyccy tcl@8.6.8%gcc - 5nus6kn ^zlib@1.2.11%gcc - - bkyl5bh zlib@1.2.8%gcc - - 64mns5m zlib@1.2.8%gcc cppflags="-O3" - - 5nus6kn zlib@1.2.11%gcc - - -Let's move on to slightly more complicated packages. ``HDF5`` is a -good example of a more complicated package, with an MPI dependency. If -we install it "out of the box," it will build with ``openmpi``. - -.. code-block:: console - - $ spack install hdf5 - ==> Installing libsigsegv - ==> Searching for binary cache of libsigsegv - ==> Finding buildcaches in /mirror/build_cache - ==> Installing libsigsegv from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/libsigsegv-2.11/linux-ubuntu16.04-x86_64-gcc-5.4.0-libsigsegv-2.11-fypapcprssrj3nstp6njprskeyynsgaz.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:08:01 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed libsigsegv from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libsigsegv-2.11-fypapcprssrj3nstp6njprskeyynsgaz - ==> Installing m4 - ==> Searching for binary cache of m4 - ==> Installing m4 from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/m4-1.4.18/linux-ubuntu16.04-x86_64-gcc-5.4.0-m4-1.4.18-suf5jtcfehivwfesrc5hjy72r4nukyel.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:24:11 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed m4 from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/m4-1.4.18-suf5jtcfehivwfesrc5hjy72r4nukyel - ==> Installing libtool - ==> Searching for binary cache of libtool - ==> Installing libtool from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/libtool-2.4.6/linux-ubuntu16.04-x86_64-gcc-5.4.0-libtool-2.4.6-o2pfwjf44353ajgr42xqtvzyvqsazkgu.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:12:47 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed libtool from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libtool-2.4.6-o2pfwjf44353ajgr42xqtvzyvqsazkgu - ==> Installing pkgconf - ==> Searching for binary cache of pkgconf - ==> Installing pkgconf from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkgconf-1.4.2/linux-ubuntu16.04-x86_64-gcc-5.4.0-pkgconf-1.4.2-fovrh7alpft646n6mhis5mml6k6e5f4v.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:00:47 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed pkgconf from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkgconf-1.4.2-fovrh7alpft646n6mhis5mml6k6e5f4v - ==> Installing util-macros - ==> Searching for binary cache of util-macros - ==> Installing util-macros from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/util-macros-1.19.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-util-macros-1.19.1-milz7fmttmptcic2qdk5cnel7ll5sybr.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:31:54 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed util-macros from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/util-macros-1.19.1-milz7fmttmptcic2qdk5cnel7ll5sybr - ==> Installing libpciaccess - ==> Searching for binary cache of libpciaccess - ==> Installing libpciaccess from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/libpciaccess-0.13.5/linux-ubuntu16.04-x86_64-gcc-5.4.0-libpciaccess-0.13.5-5urc6tcjae26fbbd2wyfohoszhgxtbmc.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:09:34 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed libpciaccess from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libpciaccess-0.13.5-5urc6tcjae26fbbd2wyfohoszhgxtbmc - ==> Installing xz - ==> Searching for binary cache of xz - ==> Installing xz from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/xz-5.2.4/linux-ubuntu16.04-x86_64-gcc-5.4.0-xz-5.2.4-teneqii2xv5u6zl5r6qi3pwurc6pmypz.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:05:03 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed xz from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/xz-5.2.4-teneqii2xv5u6zl5r6qi3pwurc6pmypz - ==> zlib is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - ==> Installing libxml2 - ==> Searching for binary cache of libxml2 - ==> Installing libxml2 from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/libxml2-2.9.8/linux-ubuntu16.04-x86_64-gcc-5.4.0-libxml2-2.9.8-wpexsphdmfayxqxd4up5vgwuqgu5woo7.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 04:56:04 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed libxml2 from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libxml2-2.9.8-wpexsphdmfayxqxd4up5vgwuqgu5woo7 - ==> Installing ncurses - ==> Searching for binary cache of ncurses - ==> Installing ncurses from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/ncurses-6.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-ncurses-6.1-3o765ourmesfrji6yeclb4wb5w54aqbh.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:04:49 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed ncurses from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/ncurses-6.1-3o765ourmesfrji6yeclb4wb5w54aqbh - ==> Installing readline - ==> Searching for binary cache of readline - ==> Installing readline from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/readline-7.0/linux-ubuntu16.04-x86_64-gcc-5.4.0-readline-7.0-nxhwrg7xwc6nbsm2v4ezwe63l6nfidbi.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:04:56 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed readline from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/readline-7.0-nxhwrg7xwc6nbsm2v4ezwe63l6nfidbi - ==> Installing gdbm - ==> Searching for binary cache of gdbm - ==> Installing gdbm from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/gdbm-1.14.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-gdbm-1.14.1-q4fpyuo7ouhkeq6d3oabtrppctpvxmes.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:18:34 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed gdbm from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gdbm-1.14.1-q4fpyuo7ouhkeq6d3oabtrppctpvxmes - ==> Installing perl - ==> Searching for binary cache of perl - ==> Installing perl from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/perl-5.26.2/linux-ubuntu16.04-x86_64-gcc-5.4.0-perl-5.26.2-ic2kyoadgp3dxfejcbllyplj2wf524fo.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:12:45 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed perl from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/perl-5.26.2-ic2kyoadgp3dxfejcbllyplj2wf524fo - ==> Installing autoconf - ==> Searching for binary cache of autoconf - ==> Installing autoconf from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/autoconf-2.69/linux-ubuntu16.04-x86_64-gcc-5.4.0-autoconf-2.69-3sx2gxeibc4oasqd4o5h6lnwpcpsgd2q.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:24:03 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed autoconf from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/autoconf-2.69-3sx2gxeibc4oasqd4o5h6lnwpcpsgd2q - ==> Installing automake - ==> Searching for binary cache of automake - ==> Installing automake from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/automake-1.16.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-automake-1.16.1-rymw7imfehycqxzj4nuy2oiw3abegooy.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:12:03 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed automake from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/automake-1.16.1-rymw7imfehycqxzj4nuy2oiw3abegooy - ==> Installing numactl - ==> Searching for binary cache of numactl - ==> Installing numactl from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/numactl-2.0.11/linux-ubuntu16.04-x86_64-gcc-5.4.0-numactl-2.0.11-ft463odrombnxlc3qew4omckhlq7tqgc.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:30:34 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed numactl from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/numactl-2.0.11-ft463odrombnxlc3qew4omckhlq7tqgc - ==> Installing hwloc - ==> Searching for binary cache of hwloc - ==> Installing hwloc from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/hwloc-1.11.9/linux-ubuntu16.04-x86_64-gcc-5.4.0-hwloc-1.11.9-43tkw5mt6huhv37vqnybqgxtkodbsava.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:08:00 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed hwloc from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hwloc-1.11.9-43tkw5mt6huhv37vqnybqgxtkodbsava - ==> Installing openmpi - ==> Searching for binary cache of openmpi - ==> Installing openmpi from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.1.3/linux-ubuntu16.04-x86_64-gcc-5.4.0-openmpi-3.1.3-3njc4q5pqdpptq6jvqjrezkffwokv2sx.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:01:54 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed openmpi from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.1.3-3njc4q5pqdpptq6jvqjrezkffwokv2sx - ==> Installing hdf5 - ==> Searching for binary cache of hdf5 - ==> Installing hdf5 from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4/linux-ubuntu16.04-x86_64-gcc-5.4.0-hdf5-1.10.4-ozyvmhzdew66byarohm4p36ep7wtcuiw.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:23:04 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed hdf5 from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4-ozyvmhzdew66byarohm4p36ep7wtcuiw - -Spack packages can also have build options, called variants. Boolean -variants can be specified using the ``+`` and ``~`` or ``-`` -sigils. There are two sigils for ``False`` to avoid conflicts with -shell parsing in different situations. Variants (boolean or otherwise) -can also be specified using the same syntax as compiler flags. Here -we can install HDF5 without MPI support. - -.. code-block:: console - - $ spack install hdf5~mpi - ==> zlib is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - ==> Installing hdf5 - ==> Searching for binary cache of hdf5 - ==> Finding buildcaches in /mirror/build_cache - ==> Installing hdf5 from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4/linux-ubuntu16.04-x86_64-gcc-5.4.0-hdf5-1.10.4-5vcv5r67vpjzenq4apyebshclelnzuja.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:23:24 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed hdf5 from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4-5vcv5r67vpjzenq4apyebshclelnzuja - -We might also want to install HDF5 with a different MPI -implementation. While MPI is not a package itself, packages can depend on -abstract interfaces like MPI. Spack handles these through "virtual -dependencies." A package, such as HDF5, can depend on the MPI -interface. Other packages (``openmpi``, ``mpich``, ``mvapich``, etc.) -provide the MPI interface. Any of these providers can be requested for -an MPI dependency. For example, we can build HDF5 with MPI support -provided by mpich by specifying a dependency on ``mpich``. Spack also -supports versioning of virtual dependencies. A package can depend on the -MPI interface at version 3, and provider packages specify what version of -the interface *they* provide. The partial spec ``^mpi@3`` can be safisfied -by any of several providers. - -.. code-block:: console - - $ spack install hdf5+hl+mpi ^mpich - ==> libsigsegv is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libsigsegv-2.11-fypapcprssrj3nstp6njprskeyynsgaz - ==> m4 is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/m4-1.4.18-suf5jtcfehivwfesrc5hjy72r4nukyel - ==> pkgconf is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkgconf-1.4.2-fovrh7alpft646n6mhis5mml6k6e5f4v - ==> ncurses is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/ncurses-6.1-3o765ourmesfrji6yeclb4wb5w54aqbh - ==> readline is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/readline-7.0-nxhwrg7xwc6nbsm2v4ezwe63l6nfidbi - ==> gdbm is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gdbm-1.14.1-q4fpyuo7ouhkeq6d3oabtrppctpvxmes - ==> perl is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/perl-5.26.2-ic2kyoadgp3dxfejcbllyplj2wf524fo - ==> autoconf is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/autoconf-2.69-3sx2gxeibc4oasqd4o5h6lnwpcpsgd2q - ==> automake is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/automake-1.16.1-rymw7imfehycqxzj4nuy2oiw3abegooy - ==> libtool is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libtool-2.4.6-o2pfwjf44353ajgr42xqtvzyvqsazkgu - ==> Installing texinfo - ==> Searching for binary cache of texinfo - ==> Finding buildcaches in /mirror/build_cache - ==> Installing texinfo from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/texinfo-6.5/linux-ubuntu16.04-x86_64-gcc-5.4.0-texinfo-6.5-zs7a2pcwhq6ho2cj2x26uxfktwkpyucn.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:18:29 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed texinfo from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/texinfo-6.5-zs7a2pcwhq6ho2cj2x26uxfktwkpyucn - ==> Installing findutils - ==> Searching for binary cache of findutils - ==> Installing findutils from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/findutils-4.6.0/linux-ubuntu16.04-x86_64-gcc-5.4.0-findutils-4.6.0-d4iajxsopzrlcjtasahxqeyjkjv5jx4v.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:07:17 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed findutils from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/findutils-4.6.0-d4iajxsopzrlcjtasahxqeyjkjv5jx4v - ==> Installing mpich - ==> Searching for binary cache of mpich - ==> Installing mpich from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/mpich-3.2.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-mpich-3.2.1-p3f7p2r5ntrynqibosglxvhwyztiwqs5.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:23:57 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed mpich from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/mpich-3.2.1-p3f7p2r5ntrynqibosglxvhwyztiwqs5 - ==> zlib is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - ==> Installing hdf5 - ==> Searching for binary cache of hdf5 - ==> Installing hdf5 from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4/linux-ubuntu16.04-x86_64-gcc-5.4.0-hdf5-1.10.4-xxd7syhgej6onpyfyewxqcqe7ltkt7ob.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:07:32 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed hdf5 from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4-xxd7syhgej6onpyfyewxqcqe7ltkt7ob - -We'll do a quick check in on what we have installed so far. - -.. code-block:: console - - $ spack find -ldf - ==> 32 installed packages - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - 6wc66et tcl@8.6.8%clang - i426yu3 ^zlib@1.2.8%clang - - i426yu3 zlib@1.2.8%clang - - 4pt75q7 zlib@1.2.11%clang - - - -- linux-ubuntu16.04-x86_64 / gcc@4.7 --------------------------- - bq2wtdx zlib@1.2.11%gcc - - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - 3sx2gxe autoconf@2.69%gcc - suf5jtc ^m4@1.4.18%gcc - fypapcp ^libsigsegv@2.11%gcc - ic2kyoa ^perl@5.26.2%gcc - q4fpyuo ^gdbm@1.14.1%gcc - nxhwrg7 ^readline@7.0%gcc - 3o765ou ^ncurses@6.1%gcc - - rymw7im automake@1.16.1%gcc - ic2kyoa ^perl@5.26.2%gcc - q4fpyuo ^gdbm@1.14.1%gcc - nxhwrg7 ^readline@7.0%gcc - 3o765ou ^ncurses@6.1%gcc - - d4iajxs findutils@4.6.0%gcc - - q4fpyuo gdbm@1.14.1%gcc - nxhwrg7 ^readline@7.0%gcc - 3o765ou ^ncurses@6.1%gcc - - 5vcv5r6 hdf5@1.10.4%gcc - 5nus6kn ^zlib@1.2.11%gcc - - ozyvmhz hdf5@1.10.4%gcc - 3njc4q5 ^openmpi@3.1.3%gcc - 43tkw5m ^hwloc@1.11.9%gcc - 5urc6tc ^libpciaccess@0.13.5%gcc - wpexsph ^libxml2@2.9.8%gcc - teneqii ^xz@5.2.4%gcc - 5nus6kn ^zlib@1.2.11%gcc - ft463od ^numactl@2.0.11%gcc - - xxd7syh hdf5@1.10.4%gcc - p3f7p2r ^mpich@3.2.1%gcc - 5nus6kn ^zlib@1.2.11%gcc - - 43tkw5m hwloc@1.11.9%gcc - 5urc6tc ^libpciaccess@0.13.5%gcc - wpexsph ^libxml2@2.9.8%gcc - teneqii ^xz@5.2.4%gcc - 5nus6kn ^zlib@1.2.11%gcc - ft463od ^numactl@2.0.11%gcc - - 5urc6tc libpciaccess@0.13.5%gcc - - fypapcp libsigsegv@2.11%gcc - - o2pfwjf libtool@2.4.6%gcc - - wpexsph libxml2@2.9.8%gcc - teneqii ^xz@5.2.4%gcc - 5nus6kn ^zlib@1.2.11%gcc - - suf5jtc m4@1.4.18%gcc - fypapcp ^libsigsegv@2.11%gcc - - p3f7p2r mpich@3.2.1%gcc - - 3o765ou ncurses@6.1%gcc - - ft463od numactl@2.0.11%gcc - - 3njc4q5 openmpi@3.1.3%gcc - 43tkw5m ^hwloc@1.11.9%gcc - 5urc6tc ^libpciaccess@0.13.5%gcc - wpexsph ^libxml2@2.9.8%gcc - teneqii ^xz@5.2.4%gcc - 5nus6kn ^zlib@1.2.11%gcc - ft463od ^numactl@2.0.11%gcc - - ic2kyoa perl@5.26.2%gcc - q4fpyuo ^gdbm@1.14.1%gcc - nxhwrg7 ^readline@7.0%gcc - 3o765ou ^ncurses@6.1%gcc - - fovrh7a pkgconf@1.4.2%gcc - - nxhwrg7 readline@7.0%gcc - 3o765ou ^ncurses@6.1%gcc - - am4pbat tcl@8.6.8%gcc - 64mns5m ^zlib@1.2.8%gcc cppflags="-O3" - - qhwyccy tcl@8.6.8%gcc - 5nus6kn ^zlib@1.2.11%gcc - - zs7a2pc texinfo@6.5%gcc - ic2kyoa ^perl@5.26.2%gcc - q4fpyuo ^gdbm@1.14.1%gcc - nxhwrg7 ^readline@7.0%gcc - 3o765ou ^ncurses@6.1%gcc - - milz7fm util-macros@1.19.1%gcc - - teneqii xz@5.2.4%gcc - - bkyl5bh zlib@1.2.8%gcc - - 64mns5m zlib@1.2.8%gcc cppflags="-O3" - - 5nus6kn zlib@1.2.11%gcc - - -Spack models the dependencies of packages as a directed acyclic graph -(DAG). The ``spack find -d`` command shows the tree representation of -that graph. We can also use the ``spack graph`` command to view the entire -DAG as a graph. - -.. code-block:: console - - $ spack graph hdf5+hl+mpi ^mpich - o hdf5 - |\ - o | zlib - / - o mpich - o findutils - |\ - | |\ - | | |\ - | | | |\ - o | | | | texinfo - | | | o | automake - | |_|/| | - |/| | | | - | | | |/ - | | | o autoconf - | |_|/| - |/| |/ - | |/| - o | | perl - o | | gdbm - o | | readline - o | | ncurses - o | | pkgconf - / / - | o libtool - |/ - o m4 - o libsigsegv - -You may also have noticed that there are some packages shown in the -``spack find -d`` output that we didn't install explicitly. These are -dependencies that were installed implicitly. A few packages installed -implicitly are not shown as dependencies in the ``spack find -d`` -output. These are build dependencies. For example, ``libpciaccess`` is a -dependency of openmpi and requires ``m4`` to build. Spack will build ``m4`` as -part of the installation of ``openmpi``, but it does not become a part of -the DAG because it is not linked in at run time. Spack handles build -dependencies differently because of their different (less strict) -consistency requirements. It is entirely possible to have two packages -using different versions of a dependency to build, which obviously cannot -be done with linked dependencies. - -``HDF5`` is more complicated than our basic example of zlib and -openssl, but it's still within the realm of software that an experienced -HPC user could reasonably expect to install given a bit of time. Now -let's look at an even more complicated package. - -.. code-block:: console - - $ spack install trilinos - ==> Installing diffutils - ==> Searching for binary cache of diffutils - ==> Finding buildcaches in /mirror/build_cache - ==> Installing diffutils from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/diffutils-3.6/linux-ubuntu16.04-x86_64-gcc-5.4.0-diffutils-3.6-2rhuivgjrna2nrxhntyde6md2khcvs34.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:30:17 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed diffutils from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/diffutils-3.6-2rhuivgjrna2nrxhntyde6md2khcvs34 - ==> Installing bzip2 - ==> Searching for binary cache of bzip2 - ==> Installing bzip2 from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/bzip2-1.0.6/linux-ubuntu16.04-x86_64-gcc-5.4.0-bzip2-1.0.6-ufczdvsqt6edesm36xiucyry7myhj7e7.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:34:37 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed bzip2 from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/bzip2-1.0.6-ufczdvsqt6edesm36xiucyry7myhj7e7 - ==> zlib is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - ==> Installing boost - ==> Searching for binary cache of boost - ==> Installing boost from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/boost-1.68.0/linux-ubuntu16.04-x86_64-gcc-5.4.0-boost-1.68.0-zbgfxapchxa4awxdwpleubfuznblxzvt.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 04:58:55 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed boost from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/boost-1.68.0-zbgfxapchxa4awxdwpleubfuznblxzvt - ==> pkgconf is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkgconf-1.4.2-fovrh7alpft646n6mhis5mml6k6e5f4v - ==> ncurses is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/ncurses-6.1-3o765ourmesfrji6yeclb4wb5w54aqbh - ==> readline is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/readline-7.0-nxhwrg7xwc6nbsm2v4ezwe63l6nfidbi - ==> gdbm is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gdbm-1.14.1-q4fpyuo7ouhkeq6d3oabtrppctpvxmes - ==> perl is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/perl-5.26.2-ic2kyoadgp3dxfejcbllyplj2wf524fo - ==> Installing openssl - ==> Searching for binary cache of openssl - ==> Installing openssl from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/openssl-1.0.2o/linux-ubuntu16.04-x86_64-gcc-5.4.0-openssl-1.0.2o-b4y3w3bsyvjla6eesv4vt6aplpfrpsha.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:24:10 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed openssl from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openssl-1.0.2o-b4y3w3bsyvjla6eesv4vt6aplpfrpsha - ==> Installing cmake - ==> Searching for binary cache of cmake - ==> Installing cmake from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.12.3/linux-ubuntu16.04-x86_64-gcc-5.4.0-cmake-3.12.3-otafqzhh4xnlq2mpakch7dr3tjfsrjnx.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:33:15 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed cmake from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.12.3-otafqzhh4xnlq2mpakch7dr3tjfsrjnx - ==> Installing glm - ==> Searching for binary cache of glm - ==> Installing glm from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/glm-0.9.7.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-glm-0.9.7.1-jnw622jwcbsymzj2fsx22omjl7tmvaws.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:30:33 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed glm from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/glm-0.9.7.1-jnw622jwcbsymzj2fsx22omjl7tmvaws - ==> libsigsegv is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libsigsegv-2.11-fypapcprssrj3nstp6njprskeyynsgaz - ==> m4 is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/m4-1.4.18-suf5jtcfehivwfesrc5hjy72r4nukyel - ==> libtool is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libtool-2.4.6-o2pfwjf44353ajgr42xqtvzyvqsazkgu - ==> util-macros is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/util-macros-1.19.1-milz7fmttmptcic2qdk5cnel7ll5sybr - ==> libpciaccess is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libpciaccess-0.13.5-5urc6tcjae26fbbd2wyfohoszhgxtbmc - ==> xz is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/xz-5.2.4-teneqii2xv5u6zl5r6qi3pwurc6pmypz - ==> libxml2 is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libxml2-2.9.8-wpexsphdmfayxqxd4up5vgwuqgu5woo7 - ==> autoconf is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/autoconf-2.69-3sx2gxeibc4oasqd4o5h6lnwpcpsgd2q - ==> automake is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/automake-1.16.1-rymw7imfehycqxzj4nuy2oiw3abegooy - ==> numactl is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/numactl-2.0.11-ft463odrombnxlc3qew4omckhlq7tqgc - ==> hwloc is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hwloc-1.11.9-43tkw5mt6huhv37vqnybqgxtkodbsava - ==> openmpi is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.1.3-3njc4q5pqdpptq6jvqjrezkffwokv2sx - ==> Installing hdf5 - ==> Searching for binary cache of hdf5 - ==> Installing hdf5 from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4/linux-ubuntu16.04-x86_64-gcc-5.4.0-hdf5-1.10.4-oqwnui7wtovuf2id4vjwcxfmxlzjus6y.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:09:10 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed hdf5 from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4-oqwnui7wtovuf2id4vjwcxfmxlzjus6y - ==> Installing openblas - ==> Searching for binary cache of openblas - ==> Installing openblas from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/openblas-0.3.3/linux-ubuntu16.04-x86_64-gcc-5.4.0-openblas-0.3.3-cyeg2yiitpuqglhvbox5gtbgsim2v5vn.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:32:04 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed openblas from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openblas-0.3.3-cyeg2yiitpuqglhvbox5gtbgsim2v5vn - ==> Installing hypre - ==> Searching for binary cache of hypre - ==> Installing hypre from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/hypre-2.15.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-hypre-2.15.1-fshksdpecwiq7r6vawfswpboedhbisju.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:07:34 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed hypre from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hypre-2.15.1-fshksdpecwiq7r6vawfswpboedhbisju - ==> Installing matio - ==> Searching for binary cache of matio - ==> Installing matio from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/matio-1.5.9/linux-ubuntu16.04-x86_64-gcc-5.4.0-matio-1.5.9-lmzdgssvobdljw52mtahelu2ju7osh6h.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:05:13 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed matio from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/matio-1.5.9-lmzdgssvobdljw52mtahelu2ju7osh6h - ==> Installing metis - ==> Searching for binary cache of metis - ==> Installing metis from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/metis-5.1.0/linux-ubuntu16.04-x86_64-gcc-5.4.0-metis-5.1.0-3wnvp4ji3wwu4v4vymszrhx6naehs6jc.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:31:42 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed metis from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/metis-5.1.0-3wnvp4ji3wwu4v4vymszrhx6naehs6jc - ==> Installing netlib-scalapack - ==> Searching for binary cache of netlib-scalapack - ==> Installing netlib-scalapack from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/netlib-scalapack-2.0.2/linux-ubuntu16.04-x86_64-gcc-5.4.0-netlib-scalapack-2.0.2-wotpfwfctgfkzzn2uescucxvvbg3tm6b.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:07:22 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed netlib-scalapack from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/netlib-scalapack-2.0.2-wotpfwfctgfkzzn2uescucxvvbg3tm6b - ==> Installing mumps - ==> Searching for binary cache of mumps - ==> Installing mumps from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/mumps-5.1.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-mumps-5.1.1-acsg2dzroox2swssgc5cwgkvdy6jcm5q.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:18:32 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed mumps from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/mumps-5.1.1-acsg2dzroox2swssgc5cwgkvdy6jcm5q - ==> Installing netcdf - ==> Searching for binary cache of netcdf - ==> Installing netcdf from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/netcdf-4.6.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-netcdf-4.6.1-mhm4izpogf4mrjidyskb6ewtzxdi7t6g.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:11:57 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed netcdf from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/netcdf-4.6.1-mhm4izpogf4mrjidyskb6ewtzxdi7t6g - ==> Installing parmetis - ==> Searching for binary cache of parmetis - ==> Installing parmetis from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/parmetis-4.0.3/linux-ubuntu16.04-x86_64-gcc-5.4.0-parmetis-4.0.3-uv6h3sqx6quqg22hxesi2mw2un3kw6b7.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:12:04 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed parmetis from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/parmetis-4.0.3-uv6h3sqx6quqg22hxesi2mw2un3kw6b7 - ==> Installing suite-sparse - ==> Searching for binary cache of suite-sparse - ==> Installing suite-sparse from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/suite-sparse-5.3.0/linux-ubuntu16.04-x86_64-gcc-5.4.0-suite-sparse-5.3.0-zaau4kifha2enpdcn3mjlrqym7hm7yon.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:22:54 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed suite-sparse from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/suite-sparse-5.3.0-zaau4kifha2enpdcn3mjlrqym7hm7yon - ==> Installing trilinos - ==> Searching for binary cache of trilinos - ==> Installing trilinos from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/trilinos-12.12.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-trilinos-12.12.1-rlsruavxqvwk2tgxzxboclbo6ykjf54r.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:18:10 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed trilinos from binary cache - -Now we're starting to see the power of Spack. Trilinos in its default -configuration has 23 top level dependecies, many of which have -dependencies of their own. Installing more complex packages can take -days or weeks even for an experienced user. Although we've done a -binary installation for the tutorial, a source installation of -trilinos using Spack takes about 3 hours (depending on the system), -but only 20 seconds of programmer time. - -Spack manages constistency of the entire DAG. Every MPI dependency will -be satisfied by the same configuration of MPI, etc. If we install -``trilinos`` again specifying a dependency on our previous HDF5 built -with ``mpich``: - -.. code-block:: console - - $ spack install trilinos +hdf5 ^hdf5+hl+mpi ^mpich - ==> diffutils is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/diffutils-3.6-2rhuivgjrna2nrxhntyde6md2khcvs34 - ==> bzip2 is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/bzip2-1.0.6-ufczdvsqt6edesm36xiucyry7myhj7e7 - ==> zlib is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - ==> boost is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/boost-1.68.0-zbgfxapchxa4awxdwpleubfuznblxzvt - ==> pkgconf is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkgconf-1.4.2-fovrh7alpft646n6mhis5mml6k6e5f4v - ==> ncurses is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/ncurses-6.1-3o765ourmesfrji6yeclb4wb5w54aqbh - ==> readline is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/readline-7.0-nxhwrg7xwc6nbsm2v4ezwe63l6nfidbi - ==> gdbm is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gdbm-1.14.1-q4fpyuo7ouhkeq6d3oabtrppctpvxmes - ==> perl is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/perl-5.26.2-ic2kyoadgp3dxfejcbllyplj2wf524fo - ==> openssl is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openssl-1.0.2o-b4y3w3bsyvjla6eesv4vt6aplpfrpsha - ==> cmake is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.12.3-otafqzhh4xnlq2mpakch7dr3tjfsrjnx - ==> glm is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/glm-0.9.7.1-jnw622jwcbsymzj2fsx22omjl7tmvaws - ==> libsigsegv is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libsigsegv-2.11-fypapcprssrj3nstp6njprskeyynsgaz - ==> m4 is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/m4-1.4.18-suf5jtcfehivwfesrc5hjy72r4nukyel - ==> autoconf is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/autoconf-2.69-3sx2gxeibc4oasqd4o5h6lnwpcpsgd2q - ==> automake is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/automake-1.16.1-rymw7imfehycqxzj4nuy2oiw3abegooy - ==> libtool is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libtool-2.4.6-o2pfwjf44353ajgr42xqtvzyvqsazkgu - ==> texinfo is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/texinfo-6.5-zs7a2pcwhq6ho2cj2x26uxfktwkpyucn - ==> findutils is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/findutils-4.6.0-d4iajxsopzrlcjtasahxqeyjkjv5jx4v - ==> mpich is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/mpich-3.2.1-p3f7p2r5ntrynqibosglxvhwyztiwqs5 - ==> hdf5 is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4-xxd7syhgej6onpyfyewxqcqe7ltkt7ob - ==> openblas is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openblas-0.3.3-cyeg2yiitpuqglhvbox5gtbgsim2v5vn - ==> Installing hypre - ==> Searching for binary cache of hypre - ==> Finding buildcaches in /mirror/build_cache - ==> Installing hypre from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/hypre-2.15.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-hypre-2.15.1-obewuozolon7tkdg4cfxc6ae2tzkronb.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:34:36 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed hypre from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hypre-2.15.1-obewuozolon7tkdg4cfxc6ae2tzkronb - ==> Installing matio - ==> Searching for binary cache of matio - ==> Installing matio from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/matio-1.5.9/linux-ubuntu16.04-x86_64-gcc-5.4.0-matio-1.5.9-gvyqldhifflmvcrtui3b6s64jcczsxxh.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:25:11 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed matio from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/matio-1.5.9-gvyqldhifflmvcrtui3b6s64jcczsxxh - ==> metis is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/metis-5.1.0-3wnvp4ji3wwu4v4vymszrhx6naehs6jc - ==> Installing netlib-scalapack - ==> Searching for binary cache of netlib-scalapack - ==> Installing netlib-scalapack from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/netlib-scalapack-2.0.2/linux-ubuntu16.04-x86_64-gcc-5.4.0-netlib-scalapack-2.0.2-p7iln2pcosw2ipyqoyr7ie6lpva2oj7r.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:32:20 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed netlib-scalapack from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/netlib-scalapack-2.0.2-p7iln2pcosw2ipyqoyr7ie6lpva2oj7r - ==> Installing mumps - ==> Searching for binary cache of mumps - ==> Installing mumps from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/mumps-5.1.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-mumps-5.1.1-cumcj5a75cagsznpjrgretxdg6okxaur.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:33:18 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed mumps from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/mumps-5.1.1-cumcj5a75cagsznpjrgretxdg6okxaur - ==> Installing netcdf - ==> Searching for binary cache of netcdf - ==> Installing netcdf from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/netcdf-4.6.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-netcdf-4.6.1-wmmx5sgwfds34v7bkkhiduar5yecrnnd.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:24:01 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed netcdf from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/netcdf-4.6.1-wmmx5sgwfds34v7bkkhiduar5yecrnnd - ==> Installing parmetis - ==> Searching for binary cache of parmetis - ==> Installing parmetis from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/parmetis-4.0.3/linux-ubuntu16.04-x86_64-gcc-5.4.0-parmetis-4.0.3-jehtatan4y2lcobj6waoqv66jj4libtz.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:07:41 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed parmetis from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/parmetis-4.0.3-jehtatan4y2lcobj6waoqv66jj4libtz - ==> suite-sparse is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/suite-sparse-5.3.0-zaau4kifha2enpdcn3mjlrqym7hm7yon - ==> Installing trilinos - ==> Searching for binary cache of trilinos - ==> Installing trilinos from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/trilinos-12.12.1/linux-ubuntu16.04-x86_64-gcc-5.4.0-trilinos-12.12.1-kqc52moweigxqxzwzfqajc6ocxwdwn4w.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:30:15 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed trilinos from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/trilinos-12.12.1-kqc52moweigxqxzwzfqajc6ocxwdwn4w - - -We see that every package in the trilinos DAG that depends on MPI now -uses ``mpich``. - -.. code-block:: console - - $ spack find -d trilinos - ==> 2 installed packages - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - trilinos@12.12.1 - ^boost@1.68.0 - ^bzip2@1.0.6 - ^zlib@1.2.11 - ^glm@0.9.7.1 - ^hdf5@1.10.4 - ^openmpi@3.1.3 - ^hwloc@1.11.9 - ^libpciaccess@0.13.5 - ^libxml2@2.9.8 - ^xz@5.2.4 - ^numactl@2.0.11 - ^hypre@2.15.1 - ^openblas@0.3.3 - ^matio@1.5.9 - ^metis@5.1.0 - ^mumps@5.1.1 - ^netlib-scalapack@2.0.2 - ^netcdf@4.6.1 - ^parmetis@4.0.3 - ^suite-sparse@5.3.0 - - trilinos@12.12.1 - ^boost@1.68.0 - ^bzip2@1.0.6 - ^zlib@1.2.11 - ^glm@0.9.7.1 - ^hdf5@1.10.4 - ^mpich@3.2.1 - ^hypre@2.15.1 - ^openblas@0.3.3 - ^matio@1.5.9 - ^metis@5.1.0 - ^mumps@5.1.1 - ^netlib-scalapack@2.0.2 - ^netcdf@4.6.1 - ^parmetis@4.0.3 - ^suite-sparse@5.3.0 - - -As we discussed before, the ``spack find -d`` command shows the -dependency information as a tree. While that is often sufficient, many -complicated packages, including trilinos, have dependencies that -cannot be fully represented as a tree. Again, the ``spack graph`` -command shows the full DAG of the dependency information. - -.. code-block:: console - - $ spack graph trilinos - o trilinos - |\ - | |\ - | | |\ - | | | |\ - | | | | |\ - | | | | | |\ - | | | | | | |\ - | | | | | | | |\ - | | | | | | | | |\ - | | | | | | | | | |\ - | | | | | | | | | | |\ - | | | | | | | | | | | |\ - | | | | | | | | | | | | |\ - o | | | | | | | | | | | | | suite-sparse - |\ \ \ \ \ \ \ \ \ \ \ \ \ \ - | |_|_|/ / / / / / / / / / / - |/| | | | | | | | | | | | | - | |\ \ \ \ \ \ \ \ \ \ \ \ \ - | | |_|_|_|_|_|/ / / / / / / - | |/| | | | | | | | | | | | - | | | |_|_|_|_|_|_|_|/ / / - | | |/| | | | | | | | | | - | | | o | | | | | | | | | parmetis - | | |/| | | | | | | | | | - | |/|/| | | | | | | | | | - | | | |/ / / / / / / / / - | | | | | | o | | | | | mumps - | |_|_|_|_|/| | | | | | - |/| | | |_|/| | | | | | - | | | |/| |/ / / / / / - | | | | |/| | | | | | - | | | | o | | | | | | netlib-scalapack - | |_|_|/| | | | | | | - |/| | |/| | | | | | | - | | |/|/ / / / / / / - | o | | | | | | | | metis - | |/ / / / / / / / - | | | | | | | o | glm - | | |_|_|_|_|/ / - | |/| | | | | | - | o | | | | | | cmake - | |\ \ \ \ \ \ \ - | o | | | | | | | openssl - | |\ \ \ \ \ \ \ \ - | | | | | o | | | | netcdf - | | |_|_|/| | | | | - | |/| | |/| | | | | - | | | | | |\ \ \ \ \ - | | | | | | | |_|/ / - | | | | | | |/| | | - | | | | | | | o | | matio - | | |_|_|_|_|/| | | - | |/| | | | |/ / / - | | | | | | | o | hypre - | |_|_|_|_|_|/| | - |/| | | | |_|/ / - | | | | |/| | | - | | | | | | o | hdf5 - | | |_|_|_|/| | - | |/| | | |/ / - | | | | |/| | - | | | | o | | openmpi - | | |_|/| | | - | |/| | | | | - | | | | |\ \ \ - | | | | | o | | hwloc - | | | | |/| | | - | | | | | |\ \ \ - | | | | | | |\ \ \ - | | | | | | o | | | libxml2 - | | |_|_|_|/| | | | - | |/| | | |/| | | | - | | | | | | | | | o boost - | | |_|_|_|_|_|_|/| - | |/| | | | | | | | - | o | | | | | | | | zlib - | / / / / / / / / - | | | | | o | | | xz - | | | | | / / / - | | | | | o | | libpciaccess - | | | | |/| | | - | | | | | |\ \ \ - | | | | | o | | | util-macros - | | | | | / / / - | | | o | | | | numactl - | | | |\ \ \ \ \ - | | | | |_|_|/ / - | | | |/| | | | - | | | | |\ \ \ \ - | | | | | |_|/ / - | | | | |/| | | - | | | | | |\ \ \ - | | | | | o | | | automake - | | |_|_|/| | | | - | |/| | | | | | | - | | | | | |/ / / - | | | | | o | | autoconf - | | |_|_|/| | | - | |/| | |/ / / - | | | |/| | | - | o | | | | | perl - | o | | | | | gdbm - | o | | | | | readline - | |/ / / / / - | o | | | | ncurses - | | |_|/ / - | |/| | | - | o | | | pkgconf - | / / / - o | | | openblas - / / / - | o | libtool - |/ / - o | m4 - o | libsigsegv - / - o bzip2 - o diffutils - -You can control how the output is displayed with a number of options. - -The ASCII output from ``spack graph`` can be difficult to parse for -complicated packages. The output can be changed to the ``graphviz`` -``.dot`` format using the ``--dot`` flag. - -.. code-block:: console - - $ spack graph --dot trilinos | dot -Tpdf trilinos_graph.pdf - -.. _basics-tutorial-uninstall: - ---------------------- -Uninstalling Packages ---------------------- - -Earlier we installed many configurations each of zlib and tcl. Now we -will go through and uninstall some of those packages that we didn't -really need. - -.. code-block:: console - - $ spack find -d tcl - ==> 3 installed packages - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - tcl@8.6.8 - ^zlib@1.2.8 - - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - tcl@8.6.8 - ^zlib@1.2.8 - - tcl@8.6.8 - ^zlib@1.2.11 - - - $ spack find zlib - ==> 6 installed packages. - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - zlib@1.2.8 zlib@1.2.11 - - -- linux-ubuntu16.04-x86_64 / gcc@4.7 --------------------------- - zlib@1.2.11 - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - zlib@1.2.8 zlib@1.2.8 zlib@1.2.11 - -We can uninstall packages by spec using the same syntax as install. - -.. code-block:: console - - $ spack uninstall zlib %gcc@4.7 - ==> The following packages will be uninstalled: - - -- linux-ubuntu16.04-x86_64 / gcc@4.7 --------------------------- - bq2wtdx zlib@1.2.11%gcc+optimize+pic+shared - - ==> Do you want to proceed? [y/N] y - ==> Successfully uninstalled zlib@1.2.11%gcc@4.7+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 /bq2wtdx - - $ spack find -lf zlib - ==> 5 installed packages. - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - i426yu3 zlib@1.2.8%clang - 4pt75q7 zlib@1.2.11%clang - - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - bkyl5bh zlib@1.2.8%gcc - 64mns5m zlib@1.2.8%gcc cppflags="-O3" - 5nus6kn zlib@1.2.11%gcc - -We can also uninstall packages by referring only to their hash. - -We can use either ``-f`` (force) or ``-R`` (remove dependents as well) to -remove packages that are required by another installed package. - -.. code-block:: console - - $ spack uninstall zlib/i426 - ==> Error: Will not uninstall zlib@1.2.8%clang@3.8.0-2ubuntu4/i426yu3 - - The following packages depend on it: - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - 6wc66et tcl@8.6.8%clang - - ==> Error: Use \`spack uninstall --dependents\` to uninstall these dependencies as well. - - $ spack uninstall -R zlib/i426 - ==> The following packages will be uninstalled: - - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - 6wc66et tcl@8.6.8%clang - i426yu3 zlib@1.2.8%clang+optimize+pic+shared - ==> Do you want to proceed? [y/N] y - ==> Successfully uninstalled tcl@8.6.8%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 /6wc66et - ==> Successfully uninstalled zlib@1.2.8%clang@3.8.0-2ubuntu4+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 /i426yu3 - -Spack will not uninstall packages that are not sufficiently -specified. The ``-a`` (all) flag can be used to uninstall multiple -packages at once. - -.. code-block:: console - - $ spack uninstall trilinos - ==> Error: trilinos matches multiple packages: - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - rlsruav trilinos@12.12.1%gcc~alloptpkgs+amesos+amesos2+anasazi+aztec+belos+boost build_type=RelWithDebInfo ~cgns~complex~dtk+epetra+epetraext+exodus+explicit_template_instantiation~float+fortran~fortrilinos+gtest+hdf5+hypre+ifpack+ifpack2~intrepid~intrepid2~isorropia+kokkos+metis~minitensor+ml+muelu+mumps~nox~openmp~phalanx~piro~pnetcdf~python~rol~rythmos+sacado~shards+shared~stk+suite-sparse~superlu~superlu-dist~teko~tempus+teuchos+tpetra~x11~xsdkflags~zlib+zoltan+zoltan2 - kqc52mo trilinos@12.12.1%gcc~alloptpkgs+amesos+amesos2+anasazi+aztec+belos+boost build_type=RelWithDebInfo ~cgns~complex~dtk+epetra+epetraext+exodus+explicit_template_instantiation~float+fortran~fortrilinos+gtest+hdf5+hypre+ifpack+ifpack2~intrepid~intrepid2~isorropia+kokkos+metis~minitensor+ml+muelu+mumps~nox~openmp~phalanx~piro~pnetcdf~python~rol~rythmos+sacado~shards+shared~stk+suite-sparse~superlu~superlu-dist~teko~tempus+teuchos+tpetra~x11~xsdkflags~zlib+zoltan+zoltan2 - - ==> Error: You can either: - a) use a more specific spec, or - b) use `spack uninstall --all` to uninstall ALL matching specs. - - - $ spack uninstall /rlsr - ==> The following packages will be uninstalled: - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - rlsruav trilinos@12.12.1%gcc~alloptpkgs+amesos+amesos2+anasazi+aztec+belos+boost build_type=RelWithDebInfo ~cgns~complex~dtk+epetra+epetraext+exodus+explicit_template_instantiation~float+fortran~fortrilinos+gtest+hdf5+hypre+ifpack+ifpack2~intrepid~intrepid2~isorropia+kokkos+metis~minitensor+ml+muelu+mumps~nox~openmp~phalanx~piro~pnetcdf~python~rol~rythmos+sacado~shards+shared~stk+suite-sparse~superlu~superlu-dist~teko~tempus+teuchos+tpetra~x11~xsdkflags~zlib+zoltan+zoltan2 - ==> Do you want to proceed? [y/N] y - ==> Successfully uninstalled trilinos@12.12.1%gcc@5.4.0~alloptpkgs+amesos+amesos2+anasazi+aztec+belos+boost build_type=RelWithDebInfo ~cgns~complex~dtk+epetra+epetraext+exodus+explicit_template_instantiation~float+fortran~fortrilinos+gtest+hdf5+hypre+ifpack+ifpack2~intrepid~intrepid2~isorropia+kokkos+metis~minitensor+ml+muelu+mumps~nox~openmp~phalanx~piro~pnetcdf~python~rol~rythmos+sacado~shards+shared~stk+suite-sparse~superlu~superlu-dist~teko~tempus+teuchos+tpetra~x11~xsdkflags~zlib+zoltan+zoltan2 arch=linux-ubuntu16.04-x86_64 /rlsruav - ------------------------------ -Advanced ``spack find`` Usage ------------------------------ - -We will go over some additional uses for the ``spack find`` command not -already covered in the :ref:`basics-tutorial-install` and -:ref:`basics-tutorial-uninstall` sections. - -The ``spack find`` command can accept what we call "anonymous specs." -These are expressions in spec syntax that do not contain a package -name. For example, ``spack find ^mpich`` will return every installed -package that depends on mpich, and ``spack find cppflags="-O3"`` will -return every package which was built with ``cppflags="-O3"``. - -.. code-block:: console - - $ spack find ^mpich - ==> 8 installed packages - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - hdf5@1.10.4 matio@1.5.9 netcdf@4.6.1 parmetis@4.0.3 - hypre@2.15.1 mumps@5.1.1 netlib-scalapack@2.0.2 trilinos@12.12.1 - - $ spack find cppflags=-O3 - ==> 1 installed packages. - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - zlib@1.2.8 - -The ``find`` command can also show which packages were installed -explicitly (rather than pulled in as a dependency) using the ``-x`` -flag. The ``-X`` flag shows implicit installs only. The ``find`` command can -also show the path to which a spack package was installed using the ``-p`` -command. - -.. code-block:: console - - $ spack find -px - ==> 10 installed packages - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - zlib@1.2.11 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/clang-3.8.0-2ubuntu4/zlib-1.2.11-4pt75q7qq6lygf3hgnona4lyc2uwedul - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - hdf5@1.10.4 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4-5vcv5r67vpjzenq4apyebshclelnzuja - hdf5@1.10.4 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4-ozyvmhzdew66byarohm4p36ep7wtcuiw - hdf5@1.10.4 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4-xxd7syhgej6onpyfyewxqcqe7ltkt7ob - tcl@8.6.8 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/tcl-8.6.8-am4pbatrtga3etyusg2akmsvrswwxno2 - tcl@8.6.8 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/tcl-8.6.8-qhwyccywhx2i6s7ob2gvjrjtj3rnfuqt - trilinos@12.12.1 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/trilinos-12.12.1-kqc52moweigxqxzwzfqajc6ocxwdwn4w - zlib@1.2.8 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.8-bkyl5bhuep6fmhuxzkmhqy25qefjcvzc - zlib@1.2.8 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.8-64mns5mvdacqvlashkf7v6lqrxixhmxu - zlib@1.2.11 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - ---------------------- -Customizing Compilers ---------------------- - - -Spack manages a list of available compilers on the system, detected -automatically from from the user's ``PATH`` variable. The ``spack -compilers`` command is an alias for the command ``spack compiler list``. - -.. code-block:: console - - $ spack compilers - ==> Available compilers - -- clang ubuntu16.04-x86_64 ------------------------------------- - clang@3.8.0-2ubuntu4 clang@3.7.1-2ubuntu2 - - -- gcc ubuntu16.04-x86_64 --------------------------------------- - gcc@5.4.0 gcc@4.7 - -The compilers are maintained in a YAML file. Later in the tutorial you -will learn how to configure compilers by hand for special cases. Spack -also has tools to add compilers, and compilers built with Spack can be -added to the configuration. - -.. code-block:: console - - $ spack install gcc @7.2.0 - ==> libsigsegv is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libsigsegv-2.11-fypapcprssrj3nstp6njprskeyynsgaz - ==> m4 is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/m4-1.4.18-suf5jtcfehivwfesrc5hjy72r4nukyel - ==> pkgconf is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkgconf-1.4.2-fovrh7alpft646n6mhis5mml6k6e5f4v - ==> ncurses is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/ncurses-6.1-3o765ourmesfrji6yeclb4wb5w54aqbh - ==> readline is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/readline-7.0-nxhwrg7xwc6nbsm2v4ezwe63l6nfidbi - ==> gdbm is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gdbm-1.14.1-q4fpyuo7ouhkeq6d3oabtrppctpvxmes - ==> perl is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/perl-5.26.2-ic2kyoadgp3dxfejcbllyplj2wf524fo - ==> autoconf is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/autoconf-2.69-3sx2gxeibc4oasqd4o5h6lnwpcpsgd2q - ==> automake is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/automake-1.16.1-rymw7imfehycqxzj4nuy2oiw3abegooy - ==> libtool is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libtool-2.4.6-o2pfwjf44353ajgr42xqtvzyvqsazkgu - ==> Installing gmp - ==> Searching for binary cache of gmp - ==> Finding buildcaches in /mirror/build_cache - ==> Installing gmp from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/gmp-6.1.2/linux-ubuntu16.04-x86_64-gcc-5.4.0-gmp-6.1.2-qc4qcfz4monpllc3nqupdo7vwinf73sw.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:18:16 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed gmp from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gmp-6.1.2-qc4qcfz4monpllc3nqupdo7vwinf73sw - ==> Installing isl - ==> Searching for binary cache of isl - ==> Installing isl from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/isl-0.18/linux-ubuntu16.04-x86_64-gcc-5.4.0-isl-0.18-vttqoutnsmjpm3ogb52rninksc7hq5ax.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:05:19 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed isl from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/isl-0.18-vttqoutnsmjpm3ogb52rninksc7hq5ax - ==> Installing mpfr - ==> Searching for binary cache of mpfr - ==> Installing mpfr from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/mpfr-3.1.6/linux-ubuntu16.04-x86_64-gcc-5.4.0-mpfr-3.1.6-jnt2nnp5pmvikbw7opueajlbwbhmjxyv.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:32:07 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed mpfr from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/mpfr-3.1.6-jnt2nnp5pmvikbw7opueajlbwbhmjxyv - ==> Installing mpc - ==> Searching for binary cache of mpc - ==> Installing mpc from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/mpc-1.1.0/linux-ubuntu16.04-x86_64-gcc-5.4.0-mpc-1.1.0-iuf3gc3zpgr4n4mditnxhff6x3joxi27.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:30:35 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed mpc from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/mpc-1.1.0-iuf3gc3zpgr4n4mditnxhff6x3joxi27 - ==> zlib is already installed in /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - Installing gcc - ==> Searching for binary cache of gcc - ==> Finding buildcaches in /mirror/build_cache - ==> Installing gcc from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0/linux-ubuntu16.04-x86_64-gcc-5.4.0-gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs.spack - ######################################################################## 100.0% - gpg: Signature made Sat Nov 10 05:22:47 2018 UTC using RSA key ID 3B7C69B2 - gpg: Good signature from "sc-tutorial (GPG created for Spack) <becker33@llnl.gov>" [unknown] - gpg: WARNING: This key is not certified with a trusted signature! - gpg: There is no indication that the signature belongs to the owner. - Primary key fingerprint: 95C7 1787 7AC0 0FFD AA8F D6E9 9CFA 4A45 3B7C 69B2 - ==> Successfully installed gcc from binary cache - [+] /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs - - $ spack find -p gcc - spack find -p gcc - ==> 1 installed package - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - gcc@7.2.0 /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs - -We can add gcc to Spack as an available compiler using the ``spack -compiler add`` command. This will allow future packages to build with -gcc@7.2.0. - -.. code-block:: console - - $ spack compiler add `spack location -i gcc@7.2.0` - ==> Added 1 new compiler to /home/ubuntu/.spack/linux/compilers.yaml - gcc@7.2.0 - ==> Compilers are defined in the following files: - /home/ubuntu/.spack/linux/compilers.yaml - -We can also remove compilers from our configuration using ``spack compiler remove <compiler_spec>`` - -.. code-block:: console - - $ spack compiler remove gcc@7.2.0 - ==> Removed compiler gcc@7.2.0 diff --git a/lib/spack/docs/tutorial_buildsystems.rst b/lib/spack/docs/tutorial_buildsystems.rst deleted file mode 100644 index 9f3785e7ee..0000000000 --- a/lib/spack/docs/tutorial_buildsystems.rst +++ /dev/null @@ -1,807 +0,0 @@ -.. Copyright 2013-2019 Lawrence Livermore National Security, LLC and other - Spack Project Developers. See the top-level COPYRIGHT file for details. - - SPDX-License-Identifier: (Apache-2.0 OR MIT) - -.. _build-systems-tutorial: - -============================== -Spack Package Build Systems -============================== - -You may begin to notice after writing a couple of package template files a -pattern emerge for some packages. For example, you may find yourself writing -an :code:`install()` method that invokes: :code:`configure`, :code:`cmake`, -:code:`make`, :code:`make install`. You may also find yourself writing -:code:`"prefix=" + prefix` as an argument to :code:`configure` or :code:`cmake`. -Rather than having you repeat these lines for all packages, Spack has -classes that can take care of these patterns. In addition, -these package files allow for finer grained control of these build systems. -In this section, we will describe each build system and give examples on -how these can be manipulated to install a package. - ------------------------ -Package Class Hierarchy ------------------------ - -.. graphviz:: - - digraph G { - - node [ - shape = "record" - ] - edge [ - arrowhead = "empty" - ] - - PackageBase -> Package [dir=back] - PackageBase -> MakefilePackage [dir=back] - PackageBase -> AutotoolsPackage [dir=back] - PackageBase -> CMakePackage [dir=back] - PackageBase -> PythonPackage [dir=back] - } - -The above diagram gives a high level view of the class hierarchy and how each -package relates. Each subclass inherits from the :code:`PackageBaseClass` -super class. The bulk of the work is done in this super class which includes -fetching, extracting to a staging directory and installing. Each subclass -then adds additional build-system-specific functionality. In the following -sections, we will go over examples of how to utilize each subclass and to see -how powerful these abstractions are when packaging. - ------------------ -Package ------------------ - -We've already seen examples of a :code:`Package` class in our walkthrough for writing -package files, so we won't be spending much time with them here. Briefly, -the Package class allows for abitrary control over the build process, whereas -subclasses rely on certain patterns (e.g. :code:`configure` :code:`make` -:code:`make install`) to be useful. :code:`Package` classes are particularly useful -for packages that have a non-conventional way of being built since the packager -can utilize some of Spack's helper functions to customize the building and -installing of a package. - -------------------- -Autotools -------------------- - -As we have seen earlier, packages using :code:`Autotools` use :code:`configure`, -:code:`make` and :code:`make install` commands to execute the build and -install process. In our :code:`Package` class, your typical build incantation will -consist of the following: - -.. code-block:: python - - def install(self, spec, prefix): - configure("--prefix=" + prefix) - make() - make("install") - -You'll see that this looks similar to what we wrote in our packaging tutorial. - -The :code:`Autotools` subclass aims to simplify writing package files and provides -convenience methods to manipulate each of the different phases for a :code:`Autotools` -build system. - -:code:`Autotools` packages consist of four phases: - -1. :code:`autoreconf()` -2. :code:`configure()` -3. :code:`build()` -4. :code:`install()` - - -Each of these phases have sensible defaults. Let's take a quick look at some -the internals of the :code:`Autotools` class: - -.. code-block:: console - - $ spack edit --build-system autotools - - -This will open the :code:`AutotoolsPackage` file in your text editor. - -.. note:: - The examples showing code for these classes is abridged to avoid having - long examples. We only show what is relevant to the packager. - - -.. literalinclude:: _spack_root/lib/spack/spack/build_systems/autotools.py - :language: python - :emphasize-lines: 33,36,54 - :lines: 30-76,240-248 - :linenos: - - -Important to note are the highlighted lines. These properties allow the -packager to set what build targets and install targets they want for their -package. If, for example, we wanted to add as our build target :code:`foo` -then we can append to our :code:`build_targets` property: - -.. code-block:: python - - build_targets = ["foo"] - -Which is similiar to invoking make in our Package - -.. code-block:: python - - make("foo") - -This is useful if we have packages that ignore environment variables and need -a command-line argument. - -Another thing to take note of is in the :code:`configure()` method. -Here we see that the :code:`prefix` argument is already included since it is a -common pattern amongst packages using :code:`Autotools`. We then only have to -override :code:`configure_args()`, which will then return it's output to -to :code:`configure()`. Then, :code:`configure()` will append the common -arguments - -Packagers also have the option to run :code:`autoreconf` in case a package -needs to update the build system and generate a new :code:`configure`. Though, -for the most part this will be unnecessary. - -Let's look at the :code:`mpileaks` package.py file that we worked on earlier: - -.. code-block:: console - - $ spack edit mpileaks - -Notice that mpileaks is a :code:`Package` class but uses the :code:`Autotools` -build system. Although this package is acceptable let's make this into an -:code:`AutotoolsPackage` class and simplify it further. - -.. literalinclude:: tutorial/examples/Autotools/0.package.py - :language: python - :emphasize-lines: 9 - :linenos: - -We first inherit from the :code:`AutotoolsPackage` class. - - -Although we could keep the :code:`install()` method, most of it can be handled -by the :code:`AutotoolsPackage` base class. In fact, the only thing that needs -to be overridden is :code:`configure_args()`. - -.. literalinclude:: tutorial/examples/Autotools/1.package.py - :language: python - :emphasize-lines: 25,26,27,28,29,30,31,32 - :linenos: - -Since Spack takes care of setting the prefix for us we can exclude that as -an argument to :code:`configure`. Our packages look simpler, and the packager -does not need to worry about whether they have properly included :code:`configure` -and :code:`make`. - -This version of the :code:`mpileaks` package installs the same as the previous, -but the :code:`AutotoolsPackage` class lets us do it with a cleaner looking -package file. - ------------------ -Makefile ------------------ - -Packages that utilize :code:`Make` or a :code:`Makefile` usually require you -to edit a :code:`Makefile` to set up platform and compiler specific variables. -These packages are handled by the :code:`Makefile` subclass which provides -convenience methods to help write these types of packages. - -A :code:`MakefilePackage` class has three phases that can be overridden. These include: - - 1. :code:`edit()` - 2. :code:`build()` - 3. :code:`install()` - -Packagers then have the ability to control how a :code:`Makefile` is edited, and -what targets to include for the build phase or install phase. - -Let's also take a look inside the :code:`MakefilePackage` class: - -.. code-block:: console - - $ spack edit --build-system makefile - -Take note of the following: - - -.. literalinclude:: _spack_root/lib/spack/spack/build_systems/makefile.py - :language: python - :lines: 14,43-61,70-88 - :emphasize-lines: 21,27,34 - :linenos: - -Similar to :code:`Autotools`, :code:`MakefilePackage` class has properties -that can be set by the packager. We can also override the different -methods highlighted. - - -Let's try to recreate the Bowtie_ package: - -.. _Bowtie: http://bowtie-bio.sourceforge.net/index.shtml - - -.. code-block:: console - - $ spack create -f https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip - ==> This looks like a URL for bowtie - ==> Found 1 version of bowtie: - - 1.2.1.1 https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip - - ==> How many would you like to checksum? (default is 1, q to abort) 1 - ==> Downloading... - ==> Fetching https://downloads.sourceforge.net/project/bowtie-bio/bowtie/1.2.1.1/bowtie-1.2.1.1-src.zip - ######################################################################## 100.0% - ==> Checksummed 1 version of bowtie - ==> This package looks like it uses the makefile build system - ==> Created template for bowtie package - ==> Created package file: /Users/mamelara/spack/var/spack/repos/builtin/packages/bowtie/package.py - -Once the fetching is completed, Spack will open up your text editor in the -usual fashion and create a template of a :code:`MakefilePackage` package.py. - -.. literalinclude:: tutorial/examples/Makefile/0.package.py - :language: python - :linenos: - -Spack was successfully able to detect that :code:`Bowtie` uses :code:`Make`. -Let's add in the rest of our details for our package: - -.. literalinclude:: tutorial/examples/Makefile/1.package.py - :language: python - :emphasize-lines: 10,11,13,14,18,20 - :linenos: - -As we mentioned earlier, most packages using a :code:`Makefile` have hard-coded -variables that must be edited. These variables are fine if you happen to not -care about setup or types of compilers used but Spack is designed to work with -any compiler. The :code:`MakefilePackage` subclass makes it easy to edit -these :code:`Makefiles` by having an :code:`edit()` method that -can be overridden. - -Let's take a look at the default :code:`Makefile` that :code:`Bowtie` provides. -If we look inside, we see that :code:`CC` and :code:`CXX` point to our GNU -compiler: - -.. code-block:: console - - $ spack stage bowtie - -.. note:: - As usual make sure you have shell support activated with spack: - :code:`source /path/to/spack_root/spack/share/spack/setup-env.sh` - -.. code-block:: console - - $ spack cd -s bowtie - $ cd bowtie-1.2 - $ vim Makefile - - -.. code-block:: make - - CPP = g++ -w - CXX = $(CPP) - CC = gcc - LIBS = $(LDFLAGS) -lz - HEADERS = $(wildcard *.h) - -To fix this, we need to use the :code:`edit()` method to write our custom -:code:`Makefile`. - -.. literalinclude:: tutorial/examples/Makefile/2.package.py - :language: python - :emphasize-lines: 23,24,25 - :linenos: - -Here we use a :code:`FileFilter` object to edit our :code:`Makefile`. It takes -in a regular expression and then replaces :code:`CC` and :code:`CXX` to whatever -Spack sets :code:`CC` and :code:`CXX` environment variables to. This allows us to -build :code:`Bowtie` with whatever compiler we specify through Spack's -:code:`spec` syntax. - -Let's change the build and install phases of our package: - -.. literalinclude:: tutorial/examples/Makefile/3.package.py - :language: python - :emphasize-lines: 28,29,30,31,32,35,36 - :linenos: - -Here demonstrate another strategy that we can use to manipulate our package -We can provide command-line arguments to :code:`make()`. Since :code:`Bowtie` -can use :code:`tbb` we can either add :code:`NO_TBB=1` as a argument to prevent -:code:`tbb` support or we can just invoke :code:`make` with no arguments. - -:code:`Bowtie` requires our :code:`install_target` to provide a path to -the install directory. We can do this by providing :code:`prefix=` as a command -line argument to :code:`make()`. - -Let's look at a couple of other examples and go through them: - -.. code-block:: console - - $ spack edit esmf - -Some packages allow environment variables to be set and will honor them. -Packages that use :code:`?=` for assignment in their :code:`Makefile` -can be set using environment variables. In our :code:`esmf` example we -set two environment variables in our :code:`edit()` method: - -.. code-block:: python - - def edit(self, spec, prefix): - for var in os.environ: - if var.startswith('ESMF_'): - os.environ.pop(var) - - # More code ... - - if self.compiler.name == 'gcc': - os.environ['ESMF_COMPILER'] = 'gfortran' - elif self.compiler.name == 'intel': - os.environ['ESMF_COMPILER'] = 'intel' - elif self.compiler.name == 'clang': - os.environ['ESMF_COMPILER'] = 'gfortranclang' - elif self.compiler.name == 'nag': - os.environ['ESMF_COMPILER'] = 'nag' - elif self.compiler.name == 'pgi': - os.environ['ESMF_COMPILER'] = 'pgi' - else: - msg = "The compiler you are building with, " - msg += "'{0}', is not supported by ESMF." - raise InstallError(msg.format(self.compiler.name)) - -As you may have noticed, we didn't really write anything to the :code:`Makefile` -but rather we set environment variables that will override variables set in -the :code:`Makefile`. - -Some packages include a configuration file that sets certain compiler variables, -platform specific variables, and the location of dependencies or libraries. -If the file is simple and only requires a couple of changes, we can overwrite -those entries with our :code:`FileFilter` object. If the configuration involves -complex changes, we can write a new configuration file from scratch. - -Let's look at an example of this in the :code:`elk` package: - -.. code-block:: console - - $ spack edit elk - -.. code-block:: python - - def edit(self, spec, prefix): - # Dictionary of configuration options - config = { - 'MAKE': 'make', - 'AR': 'ar' - } - - # Compiler-specific flags - flags = '' - if self.compiler.name == 'intel': - flags = '-O3 -ip -unroll -no-prec-div' - elif self.compiler.name == 'gcc': - flags = '-O3 -ffast-math -funroll-loops' - elif self.compiler.name == 'pgi': - flags = '-O3 -lpthread' - elif self.compiler.name == 'g95': - flags = '-O3 -fno-second-underscore' - elif self.compiler.name == 'nag': - flags = '-O4 -kind=byte -dusty -dcfuns' - elif self.compiler.name == 'xl': - flags = '-O3' - config['F90_OPTS'] = flags - config['F77_OPTS'] = flags - - # BLAS/LAPACK support - # Note: BLAS/LAPACK must be compiled with OpenMP support - # if the +openmp variant is chosen - blas = 'blas.a' - lapack = 'lapack.a' - if '+blas' in spec: - blas = spec['blas'].libs.joined() - if '+lapack' in spec: - lapack = spec['lapack'].libs.joined() - # lapack must come before blas - config['LIB_LPK'] = ' '.join([lapack, blas]) - - # FFT support - if '+fft' in spec: - config['LIB_FFT'] = join_path(spec['fftw'].prefix.lib, - 'libfftw3.so') - config['SRC_FFT'] = 'zfftifc_fftw.f90' - else: - config['LIB_FFT'] = 'fftlib.a' - config['SRC_FFT'] = 'zfftifc.f90' - - # MPI support - if '+mpi' in spec: - config['F90'] = spec['mpi'].mpifc - config['F77'] = spec['mpi'].mpif77 - else: - config['F90'] = spack_fc - config['F77'] = spack_f77 - config['SRC_MPI'] = 'mpi_stub.f90' - - # OpenMP support - if '+openmp' in spec: - config['F90_OPTS'] += ' ' + self.compiler.openmp_flag - config['F77_OPTS'] += ' ' + self.compiler.openmp_flag - else: - config['SRC_OMP'] = 'omp_stub.f90' - - # Libxc support - if '+libxc' in spec: - config['LIB_libxc'] = ' '.join([ - join_path(spec['libxc'].prefix.lib, 'libxcf90.so'), - join_path(spec['libxc'].prefix.lib, 'libxc.so') - ]) - config['SRC_libxc'] = ' '.join([ - 'libxc_funcs.f90', - 'libxc.f90', - 'libxcifc.f90' - ]) - else: - config['SRC_libxc'] = 'libxcifc_stub.f90' - - # Write configuration options to include file - with open('make.inc', 'w') as inc: - for key in config: - inc.write('{0} = {1}\n'.format(key, config[key])) - -:code:`config` is just a dictionary that we can add key-value pairs to. By the -end of the :code:`edit()` method we write the contents of our dictionary to -:code:`make.inc`. - ---------------- -CMake ---------------- - -CMake_ is another common build system that has been gaining popularity. It works -in a similar manner to :code:`Autotools` but with differences in variable names, -the number of configuration options available, and the handling of shared libraries. -Typical build incantations look like this: - -.. _CMake: https://cmake.org - -.. code-block:: python - - def install(self, spec, prefix): - cmake("-DCMAKE_INSTALL_PREFIX:PATH=/path/to/install_dir ..") - make() - make("install") - -As you can see from the example above, it's very similar to invoking -:code:`configure` and :code:`make` in an :code:`Autotools` build system. However, -the variable names and options differ. Most options in CMake are prefixed -with a :code:`'-D'` flag to indicate a configuration setting. - -In the :code:`CMakePackage` class we can override the following phases: - -1. :code:`cmake()` -2. :code:`build()` -3. :code:`install()` - -The :code:`CMakePackage` class also provides sensible defaults so we only need to -override :code:`cmake_args()`. - -Let's look at these defaults in the :code:`CMakePackage` class in the :code:`_std_args()` method: - -.. code-block:: console - - $ spack edit --build-system cmake - -.. literalinclude:: _spack_root/lib/spack/spack/build_systems/cmake.py - :language: python - :lines: 102-147 - :emphasize-lines: 10,18,24,36,37,38,44 - :linenos: - -Some :code:`CMake` packages use different generators. Spack is able to support -Unix-Makefile_ generators as well as Ninja_ generators. - -.. _Unix-Makefile: https://cmake.org/cmake/help/v3.4/generator/Unix%20Makefiles.html -.. _Ninja: https://cmake.org/cmake/help/v3.4/generator/Ninja.html - -If no generator is specified Spack will default to :code:`Unix Makefiles`. - -Next we setup the build type. In :code:`CMake` you can specify the build type -that you want. Options include: - -1. :code:`empty` -2. :code:`Debug` -3. :code:`Release` -4. :code:`RelWithDebInfo` -5. :code:`MinSizeRel` - -With these options you can specify whether you want your executable to have -the debug version only, release version or the release with debug information. -Release executables tend to be more optimized than Debug. In Spack, we set -the default as RelWithDebInfo unless otherwise specified through a variant. - -Spack then automatically sets up the :code:`-DCMAKE_INSTALL_PREFIX` path, -appends the build type (:code:`RelWithDebInfo` default), and then specifies a verbose -:code:`Makefile`. - -Next we add the :code:`rpaths` to :code:`-DCMAKE_INSTALL_RPATH:STRING`. - - -Finally we add to :code:`-DCMAKE_PREFIX_PATH:STRING` the locations of all our -dependencies so that :code:`CMake` can find them. - -In the end our :code:`cmake` line will look like this (example is :code:`xrootd`): - -.. code-block:: console - - $ cmake $HOME/spack/var/spack/stage/xrootd-4.6.0-4ydm74kbrp4xmcgda5upn33co5pwddyk/xrootd-4.6.0 -G Unix Makefiles -DCMAKE_INSTALL_PREFIX:PATH=$HOME/spack/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/xrootd-4.6.0-4ydm74kbrp4xmcgda5upn33co5pwddyk -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCMAKE_FIND_FRAMEWORK:STRING=LAST -DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=FALSE -DCMAKE_INSTALL_RPATH:STRING=$HOME/spack/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/xrootd-4.6.0-4ydm74kbrp4xmcgda5upn33co5pwddyk/lib:$HOME/spack/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/xrootd-4.6.0-4ydm74kbrp4xmcgda5upn33co5pwddyk/lib64 -DCMAKE_PREFIX_PATH:STRING=$HOME/spack/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/cmake-3.9.4-hally3vnbzydiwl3skxcxcbzsscaasx5 - -We can see now how :code:`CMake` takes care of a lot of the boilerplate code -that would have to be otherwise typed in. - -Let's try to recreate callpath_: - -.. _callpath: https://github.com/LLNL/callpath.git - -.. code-block:: console - - $ spack create -f https://github.com/llnl/callpath/archive/v1.0.3.tar.gz - ==> This looks like a URL for callpath - ==> Found 4 versions of callpath: - - 1.0.3 https://github.com/LLNL/callpath/archive/v1.0.3.tar.gz - 1.0.2 https://github.com/LLNL/callpath/archive/v1.0.2.tar.gz - 1.0.1 https://github.com/LLNL/callpath/archive/v1.0.1.tar.gz - 1.0 https://github.com/LLNL/callpath/archive/v1.0.tar.gz - - ==> How many would you like to checksum? (default is 1, q to abort) 1 - ==> Downloading... - ==> Fetching https://github.com/LLNL/callpath/archive/v1.0.3.tar.gz - ######################################################################## 100.0% - ==> Checksummed 1 version of callpath - ==> This package looks like it uses the cmake build system - ==> Created template for callpath package - ==> Created package file: /Users/mamelara/spack/var/spack/repos/builtin/packages/callpath/package.py - - -which then produces the following template: - -.. literalinclude:: tutorial/examples/Cmake/0.package.py - :language: python - :linenos: - -Again we fill in the details: - -.. literalinclude:: tutorial/examples/Cmake/1.package.py - :language: python - :linenos: - :emphasize-lines: 9,13,14,18,19,20,21,22,23 - -As mentioned earlier, Spack will use sensible defaults to prevent repeated code -and to make writing :code:`CMake` package files simpler. - -In callpath, we want to add options to :code:`CALLPATH_WALKER` as well as add -compiler flags. We add the following options like so: - -.. literalinclude:: tutorial/examples/Cmake/2.package.py - :language: python - :linenos: - :emphasize-lines: 26,30,31 - -Now we can control our build options using :code:`cmake_args()`. If defaults are -sufficient enough for the package, we can leave this method out. - -:code:`CMakePackage` classes allow for control of other features in the -build system. For example, you can specify the path to the "out of source" -build directory and also point to the root of the :code:`CMakeLists.txt` file if it -is placed in a non-standard location. - -A good example of a package that has its :code:`CMakeLists.txt` file located at a -different location is found in :code:`spades`. - -.. code-block:: console - - $ spack edit spades - -.. code-block:: python - - root_cmakelists_dir = "src" - -Here :code:`root_cmakelists_dir` will tell Spack where to find the location -of :code:`CMakeLists.txt`. In this example, it is located a directory level below in -the :code:`src` directory. - -Some :code:`CMake` packages also require the :code:`install` phase to be -overridden. For example, let's take a look at :code:`sniffles`. - -.. code-block:: console - - $ spack edit sniffles - -In the :code:`install()` method, we have to manually install our targets -so we override the :code:`install()` method to do it for us: - -.. code-block:: python - - # the build process doesn't actually install anything, do it by hand - def install(self, spec, prefix): - mkdir(prefix.bin) - src = "bin/sniffles-core-{0}".format(spec.version.dotted) - binaries = ['sniffles', 'sniffles-debug'] - for b in binaries: - install(join_path(src, b), join_path(prefix.bin, b)) - - --------------- -PythonPackage --------------- - -Python extensions and modules are built differently from source than most -applications. Python uses a :code:`setup.py` script to install Python modules. -The script consists of a call to :code:`setup()` which provides the information -required to build a module to Distutils. If you're familiar with pip or -easy_install, setup.py does the same thing. - -These modules are usually installed using the following line: - -.. code-block:: console - - $ python setup.py install - -There are also a list of commands and phases that you can call. To see the full -list you can run: - -.. code-block:: console - - $ python setup.py --help-commands - Standard commands: - build build everything needed to install - build_py "build" pure Python modules (copy to build directory) - build_ext build C/C++ extensions (compile/link to build directory) - build_clib build C/C++ libraries used by Python extensions - build_scripts "build" scripts (copy and fixup #! line) - clean (no description available) - install install everything from build directory - install_lib install all Python modules (extensions and pure Python) - install_headers install C/C++ header files - install_scripts install scripts (Python or otherwise) - install_data install data files - sdist create a source distribution (tarball, zip file, etc.) - register register the distribution with the Python package index - bdist create a built (binary) distribution - bdist_dumb create a "dumb" built distribution - bdist_rpm create an RPM distribution - bdist_wininst create an executable installer for MS Windows - upload upload binary package to PyPI - check perform some checks on the package - - -We can write package files for Python packages using the :code:`Package` class, -but the class brings with it a lot of methods that are useless for Python packages. -Instead, Spack has a :code:`PythonPackage` subclass that allows packagers -of Python modules to be able to invoke :code:`setup.py` and use :code:`Distutils`, -which is much more familiar to a typical python user. - -To see the defaults that Spack has for each a methods, we will take a look -at the :code:`PythonPackage` class: - -.. code-block:: console - - $ spack edit --build-system python - -We see the following: - - -.. literalinclude:: _spack_root/lib/spack/spack/build_systems/python.py - :language: python - :lines: 19,146-357 - :linenos: - -Each of these methods have sensible defaults or they can be overridden. - -We will write a package file for Pandas_: - -.. _pandas: https://pandas.pydata.org - -.. code-block:: console - - $ spack create -f https://pypi.io/packages/source/p/pandas/pandas-0.19.0.tar.gz - ==> This looks like a URL for pandas - ==> Warning: Spack was unable to fetch url list due to a certificate verification problem. You can try running spack -k, which will not check SSL certificates. Use this at your own risk. - ==> Found 1 version of pandas: - - 0.19.0 https://pypi.io/packages/source/p/pandas/pandas-0.19.0.tar.gz - - ==> How many would you like to checksum? (default is 1, q to abort) 1 - ==> Downloading... - ==> Fetching https://pypi.io/packages/source/p/pandas/pandas-0.19.0.tar.gz - ######################################################################## 100.0% - ==> Checksummed 1 version of pandas - ==> This package looks like it uses the python build system - ==> Changing package name from pandas to py-pandas - ==> Created template for py-pandas package - ==> Created package file: /Users/mamelara/spack/var/spack/repos/builtin/packages/py-pandas/package.py - -And we are left with the following template: - -.. literalinclude:: tutorial/examples/PyPackage/0.package.py - :language: python - :linenos: - -As you can see this is not any different than any package template that we have -written. We have the choice of providing build options or using the sensible -defaults - -Luckily for us, there is no need to provide build args. - -Next we need to find the dependencies of a package. Dependencies are usually -listed in :code:`setup.py`. You can find the dependencies by searching for -:code:`install_requires` keyword in that file. Here it is for :code:`Pandas`: - -.. code-block:: python - - # ... code - if sys.version_info[0] >= 3: - - setuptools_kwargs = { - 'zip_safe': False, - 'install_requires': ['python-dateutil >= 2', - 'pytz >= 2011k', - 'numpy >= %s' % min_numpy_ver], - 'setup_requires': ['numpy >= %s' % min_numpy_ver], - } - if not _have_setuptools: - sys.exit("need setuptools/distribute for Py3k" - "\n$ pip install distribute") - - # ... more code - -You can find a more comprehensive list at the Pandas documentation_. - -.. _documentation: https://pandas.pydata.org/pandas-docs/stable/install.html - - -By reading the documentation and :code:`setup.py` we found that :code:`Pandas` -depends on :code:`python-dateutil`, :code:`pytz`, and :code:`numpy`, :code:`numexpr`, -and finally :code:`bottleneck`. - -Here is the completed :code:`Pandas` script: - -.. literalinclude:: tutorial/examples/PyPackage/1.package.py - :language: python - :linenos: - -It is quite important to declare all the dependencies of a Python package. -Spack can "activate" Python packages to prevent the user from having to -load each dependency module explictly. If a dependency is missed, Spack will -be unable to properly activate the package and it will cause an issue. To -learn more about extensions go to :ref:`cmd-spack-extensions`. - -From this example, you can see that building Python modules is made easy -through the :code:`PythonPackage` class. - -------------------- -Other Build Systems -------------------- - -Although we won't get in depth with any of the other build systems that Spack -supports, it is worth mentioning that Spack does provide subclasses -for the following build systems: - -1. :code:`IntelPackage` -2. :code:`SconsPackage` -3. :code:`WafPackage` -4. :code:`RPackage` -5. :code:`PerlPackage` -6. :code:`QMakePackage` - - -Each of these classes have their own abstractions to help assist in writing -package files. For whatever doesn't fit nicely into the other build-systems, -you can use the :code:`Package` class. - -Hopefully by now you can see how we aim to make packaging simple and -robust through these classes. If you want to learn more about these build -systems, check out :ref:`installation_procedure` in the Packaging Guide. diff --git a/lib/spack/docs/tutorial_configuration.rst b/lib/spack/docs/tutorial_configuration.rst deleted file mode 100644 index 879bb94967..0000000000 --- a/lib/spack/docs/tutorial_configuration.rst +++ /dev/null @@ -1,951 +0,0 @@ -.. Copyright 2013-2019 Lawrence Livermore National Security, LLC and other - Spack Project Developers. See the top-level COPYRIGHT file for details. - - SPDX-License-Identifier: (Apache-2.0 OR MIT) - -.. _configs-tutorial: - -====================== -Configuration Tutorial -====================== - -This tutorial will guide you through various configuration options -that allow you to customize Spack's behavior with respect to -software installation. We will first cover the configuration file -hierarchy. Then, we will cover configuration options for compilers, -focusing on how they can be used to extend Spack's compiler auto-detection. -Next, we will cover the packages configuration file, focusing on -how it can be used to override default build options as well as -specify external package installations to use. Finally, we will -briefly touch on the config configuration file, which manages more -high-level Spack configuration options. - -For all of these features, we will demonstrate how we build up a full -configuration file. For some, we will then demonstrate how the -configuration affects the install command, and for others we will use -the ``spack spec`` command to demonstrate how the configuration -changes have affected Spack's concretization algorithm. The provided -output is all from a server running Ubuntu version 16.04. - -.. _configs-tutorial-scopes: - --------------------- -Configuration Scopes --------------------- - -Depending on your use case, you may want to provide configuration -settings common to everyone on your team, or you may want to set -default behaviors specific to a single user account. Spack provides -six configuration *scopes* to handle this customization. These scopes, -in order of decreasing priority, are: - -============ =================================================== -Scope Directory -============ =================================================== -Command-line N/A -Custom Custom directory, specified with ``--config-scope`` -User ``~/.spack/`` -Site ``$SPACK_ROOT/etc/spack/`` -System ``/etc/spack/`` -Defaults ``$SPACK_ROOT/etc/spack/defaults/`` -============ =================================================== - -Spack's default configuration settings reside in -``$SPACK_ROOT/etc/spack/defaults``. These are useful for reference, -but should never be directly edited. To override these settings, -create new configuration files in any of the higher-priority -configuration scopes. - -A particular cluster may have multiple Spack installations associated -with different projects. To provide settings common to all Spack -installations, put your configuration files in ``/etc/spack``. -To provide settings specific to a particular Spack installation, -you can use the ``$SPACK_ROOT/etc/spack`` directory. - -For settings specific to a particular user, you will want to add -configuration files to the ``~/.spack`` directory. When Spack first -checked for compilers on your system, you may have noticed that it -placed your compiler configuration in this directory. - -Configuration settings can also be placed in a custom location, -which is then specified on the command line via ``--config-scope``. -An example use case is managing two sets of configurations, one for -development and another for production preferences. - -Settings specified on the command line have precedence over all -other configuration scopes. - -You can also use ``spack config blame <config>`` for displaying -the effective configuration. Spack will show from which scopes -the configuration has been assembled. - -^^^^^^^^^^^^^^^^^^^^^^^^ -Platform-specific Scopes -^^^^^^^^^^^^^^^^^^^^^^^^ - -Some facilities manage multiple platforms from a single shared -file system. In order to handle this, each of the configuration -scopes listed above has two *sub-scopes*: platform-specific and -platform-independent. For example, compiler settings can be stored -in ``compilers.yaml`` configuration files in the following locations: - -#. ``~/.spack/<platform>/compilers.yaml`` -#. ``~/.spack/compilers.yaml`` -#. ``$SPACK_ROOT/etc/spack/<platform>/compilers.yaml`` -#. ``$SPACK_ROOT/etc/spack/compilers.yaml`` -#. ``/etc/spack/<platform>/compilers.yaml`` -#. ``/etc/spack/compilers.yaml`` -#. ``$SPACK_ROOT/etc/defaults/<platform>/compilers.yaml`` -#. ``$SPACK_ROOT/etc/defaults/compilers.yaml`` - -These files are listed in decreasing order of precedence, so files in -``~/.spack/<platform>`` will override settings in ``~/.spack``. - ------------ -YAML Format ------------ - -Spack configurations are YAML dictionaries. Every configuration file -begins with a top-level dictionary that tells Spack which -configuration set it modifies. When Spack checks its configuration, -the configuration scopes are updated as dictionaries in increasing -order of precedence, allowing higher precedence files to override -lower. YAML dictionaries use a colon ":" to specify key-value -pairs. Spack extends YAML syntax slightly to allow a double-colon -"::" to specify a key-value pair. When a double-colon is used to -specify a key-value pair, instead of adding that section, Spack -replaces what was in that section with the new value. For example, -consider a user's compilers configuration file as follows: - -.. code-block:: yaml - - compilers:: - - compiler: - environment: {} - extra_rpaths: [] - flags: {} - modules: [] - operating_system: ubuntu16.04 - paths: - cc: /usr/bin/gcc - cxx: /usr/bin/g++ - f77: /usr/bin/gfortran - fc: /usr/bin/gfortran - spec: gcc@5.4.0 - target: x86_64 - - -This ensures that no other compilers are used, as the user configuration -scope is the last scope searched and the ``compilers::`` line replaces -all previous configuration files information. If the same -configuration file had a single colon instead of the double colon, it -would add the GCC version 5.4.0 compiler to whatever other compilers -were listed in other configuration files. - -.. _configs-tutorial-compilers: - ----------------------- -Compiler Configuration ----------------------- - -For most tasks, we can use Spack with the compilers auto-detected the -first time Spack runs on a system. As discussed in the basic -installation tutorial, we can also tell Spack where compilers are -located using the ``spack compiler add`` command. However, in some -circumstances we want even more fine-grained control over the -compilers available. This section will teach you how to exercise that -control using the compilers configuration file. - -We will start by opening the compilers configuration file: - -.. code-block:: console - - $ spack config edit compilers - - -.. code-block:: yaml - - compilers: - - compiler: - environment: {} - extra_rpaths: [] - flags: {} - modules: [] - operating_system: ubuntu16.04 - paths: - cc: /usr/bin/clang-3.7 - cxx: /usr/bin/clang++-3.7 - f77: null - fc: null - spec: clang@3.7.1-2ubuntu2 - target: x86_64 - - compiler: - environment: {} - extra_rpaths: [] - flags: {} - modules: [] - operating_system: ubuntu16.04 - paths: - cc: /usr/bin/clang - cxx: /usr/bin/clang++ - f77: null - fc: null - spec: clang@3.8.0-2ubuntu4 - target: x86_64 - - compiler: - environment: {} - extra_rpaths: [] - flags: {} - modules: [] - operating_system: ubuntu16.04 - paths: - cc: /usr/bin/gcc-4.7 - cxx: /usr/bin/g++-4.7 - f77: /usr/bin/gfortran-4.7 - fc: /usr/bin/gfortran-4.7 - spec: gcc@4.7 - target: x86_64 - - compiler: - environment: {} - extra_rpaths: [] - flags: {} - modules: [] - operating_system: ubuntu16.04 - paths: - cc: /usr/bin/gcc - cxx: /usr/bin/g++ - f77: /usr/bin/gfortran - fc: /usr/bin/gfortran - spec: gcc@5.4.0 - target: x86_64 - - -This specifies two versions of the GCC compiler and two versions of the -Clang compiler with no Flang compiler. Now suppose we have a code that -we want to compile with the Clang compiler for C/C++ code, but with -gfortran for Fortran components. We can do this by adding another entry -to the ``compilers.yaml`` file. - -.. code-block:: yaml - - - compiler: - environment: {} - extra_rpaths: [] - flags: {} - modules: [] - operating_system: ubuntu16.04 - paths: - cc: /usr/bin/clang - cxx: /usr/bin/clang++ - f77: /usr/bin/gfortran - fc: /usr/bin/gfortran - spec: clang@3.8.0-gfortran - target: x86_64 - - -Let's talk about the sections of this compiler entry that we've changed. -The biggest change we've made is to the ``paths`` section. This lists -the paths to the compilers to use for each language/specification. -In this case, we point to the Clang compiler for C/C++ and the gfortran -compiler for both specifications of Fortran. We've also changed the -``spec`` entry for this compiler. The ``spec`` entry is effectively the -name of the compiler for Spack. It consists of a name and a version -number, separated by the ``@`` sigil. The name must be one of the supported -compiler names in Spack (gcc, intel, pgi, xl, xl_r, clang, nag, cce, arm). -The version number can be an arbitrary string of alphanumeric characters, -as well as ``-``, ``.``, and ``_``. The ``target`` and ``operating_system`` -sections we leave unchanged. These sections specify when Spack can use -different compilers, and are primarily useful for configuration files that -will be used across multiple systems. - -We can verify that our new compiler works by invoking it now: - -.. code-block:: console - - $ spack install --no-cache zlib %clang@3.8.0-gfortran - ... - - -This new compiler also works on Fortran codes: - -.. code-block:: console - - $ spack install --no-cache cfitsio~bzip2 %clang@3.8.0-gfortran - ... - - -^^^^^^^^^^^^^^ -Compiler Flags -^^^^^^^^^^^^^^ - -Some compilers may require specific compiler flags to work properly in -a particular computing environment. Spack provides configuration -options for setting compiler flags every time a specific compiler is -invoked. These flags become part of the package spec and therefore of -the build provenance. As on the command line, the flags are set -through the implicit build variables ``cflags``, ``cxxflags``, ``cppflags``, -``fflags``, ``ldflags``, and ``ldlibs``. - -Let's open our compilers configuration file again and add a compiler flag: - -.. code-block:: yaml - - - compiler: - environment: {} - extra_rpaths: [] - flags: - cppflags: -g - modules: [] - operating_system: ubuntu16.04 - paths: - cc: /usr/bin/clang - cxx: /usr/bin/clang++ - f77: /usr/bin/gfortran - fc: /usr/bin/gfortran - spec: clang@3.8.0-gfortran - target: x86_64 - - -We can test this out using the ``spack spec`` command to show how the -spec is concretized: - -.. code-block:: console - - $ spack spec cfitsio %clang@3.8.0-gfortran - Input spec - -------------------------------- - cfitsio%clang@3.8.0-gfortran - - Normalized - -------------------------------- - cfitsio%clang@3.8.0-gfortran - - Concretized - -------------------------------- - cfitsio@3.410%clang@3.8.0-gfortran cppflags="-g" +bzip2+shared arch=linux-ubuntu16.04-x86_64 - ^bzip2@1.0.6%clang@3.8.0-gfortran cppflags="-g" +shared arch=linux-ubuntu16.04-x86_64 - - -We can see that ``cppflags="-g"`` has been added to every node in the DAG. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Advanced Compiler Configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -There are three fields of the compiler configuration entry that we -have not yet talked about. - -The ``modules`` field of the compiler is used primarily on Cray systems, -but can be useful on any system that has compilers that are only -useful when a particular module is loaded. Any modules in the -``modules`` field of the compiler configuration will be loaded as part -of the build environment for packages using that compiler. - -The ``extra_rpaths`` field of the compiler configuration is used for -compilers that do not rpath all of their dependencies by -default. Since compilers are often installed externally to Spack, -Spack is unable to manage compiler dependencies and enforce -rpath usage. This can lead to packages not finding link dependencies -imposed by the compiler properly. For compilers that impose link -dependencies on the resulting executables that are not rpath'ed into -the executable automatically, the ``extra_rpaths`` field of the compiler -configuration tells Spack which dependencies to rpath into every -executable created by that compiler. The executables will then be able -to find the link dependencies imposed by the compiler. As an example, -this field can be set by: - -.. code-block:: yaml - - - compiler: - ... - extra_rpaths: - - /apps/intel/ComposerXE2017/compilers_and_libraries_2017.5.239/linux/compiler/lib/intel64_lin - ... - - -The ``environment`` field of the compiler configuration is used for -compilers that require environment variables to be set during build -time. For example, if your Intel compiler suite requires the -``INTEL_LICENSE_FILE`` environment variable to point to the proper -license server, you can set this in ``compilers.yaml`` as follows: - -.. code-block:: yaml - - - compiler: - environment: - set: - INTEL_LICENSE_FILE: 1713@license4 - ... - - -In addition to ``set``, ``environment`` also supports ``unset``, -``prepend-path``, and ``append-path``. - -.. _configs-tutorial-package-prefs: - -------------------------------- -Configuring Package Preferences -------------------------------- - -Package preferences in Spack are managed through the ``packages.yaml`` -configuration file. First, we will look at the default -``packages.yaml`` file. - -.. code-block:: console - - $ spack config --scope defaults edit packages - - -.. literalinclude:: _spack_root/etc/spack/defaults/packages.yaml - :language: yaml - - -This sets the default preferences for compilers and for providers of -virtual packages. To illustrate how this works, suppose we want to -change the preferences to prefer the Clang compiler and to prefer -MPICH over OpenMPI. Currently, we prefer GCC and OpenMPI. - -.. code-block:: console - - $ spack spec hdf5 - Input spec - -------------------------------- - hdf5 - - Concretized - -------------------------------- - hdf5@1.10.4%gcc@5.4.0~cxx~debug~fortran~hl+mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - ^openmpi@3.1.3%gcc@5.4.0~cuda+cxx_exceptions fabrics= ~java~legacylaunchers~memchecker~pmi schedulers= ~sqlite3~thread_multiple+vt arch=linux-ubuntu16.04-x86_64 - ^hwloc@1.11.9%gcc@5.4.0~cairo~cuda+libxml2+pci+shared arch=linux-ubuntu16.04-x86_64 - ^libpciaccess@0.13.5%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^libtool@2.4.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^m4@1.4.18%gcc@5.4.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - ^libsigsegv@2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^pkgconf@1.4.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^util-macros@1.19.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^libxml2@2.9.8%gcc@5.4.0~python arch=linux-ubuntu16.04-x86_64 - ^xz@5.2.4%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^zlib@1.2.11%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 - ^numactl@2.0.11%gcc@5.4.0 patches=592f30f7f5f757dfc239ad0ffd39a9a048487ad803c26b419e0f96b8cda08c1a arch=linux-ubuntu16.04-x86_64 - ^autoconf@2.69%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^perl@5.26.2%gcc@5.4.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - ^gdbm@1.14.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^readline@7.0%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^ncurses@6.1%gcc@5.4.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - ^automake@1.16.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - - -Now we will open the packages configuration file and update our -preferences. - -.. code-block:: console - - $ spack config edit packages - - -.. code-block:: yaml - - packages: - all: - compiler: [clang, gcc, intel, pgi, xl, nag] - providers: - mpi: [mpich, openmpi] - - -Because of the configuration scoping we discussed earlier, this -overrides the default settings just for these two items. - -.. code-block:: console - - $ spack spec hdf5 - Input spec - -------------------------------- - hdf5 - - Concretized - -------------------------------- - hdf5@1.10.4%clang@3.8.0-2ubuntu4~cxx~debug~fortran~hl+mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - ^mpich@3.2.1%clang@3.8.0-2ubuntu4 device=ch3 +hydra netmod=tcp +pmi+romio~verbs arch=linux-ubuntu16.04-x86_64 - ^findutils@4.6.0%clang@3.8.0-2ubuntu4 patches=84b916c0bf8c51b7e7b28417692f0ad3e7030d1f3c248ba77c42ede5c1c5d11e,bd9e4e5cc280f9753ae14956c4e4aa17fe7a210f55dd6c84aa60b12d106d47a2 arch=linux-ubuntu16.04-x86_64 - ^autoconf@2.69%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^m4@1.4.18%clang@3.8.0-2ubuntu4 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - ^libsigsegv@2.11%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^perl@5.26.2%clang@3.8.0-2ubuntu4+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - ^gdbm@1.14.1%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^readline@7.0%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^ncurses@6.1%clang@3.8.0-2ubuntu4~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - ^pkgconf@1.4.2%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^automake@1.16.1%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^libtool@2.4.6%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^texinfo@6.5%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^zlib@1.2.11%clang@3.8.0-2ubuntu4+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 - - -^^^^^^^^^^^^^^^^^^^ -Variant Preferences -^^^^^^^^^^^^^^^^^^^ - -The packages configuration file can also set variant preferences for -package variants. For example, let's change our preferences to build all -packages without shared libraries. We will accomplish this by turning -off the ``shared`` variant on all packages that have one. - -.. code-block:: yaml - - packages: - all: - compiler: [clang, gcc, intel, pgi, xl, nag] - providers: - mpi: [mpich, openmpi] - variants: ~shared - - -We can check the effect of this command with ``spack spec hdf5`` again. - -.. code-block:: console - - $ spack spec hdf5 - Input spec - -------------------------------- - hdf5 - - Concretized - -------------------------------- - hdf5@1.10.4%clang@3.8.0-2ubuntu4~cxx~debug~fortran~hl+mpi+pic~shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - ^mpich@3.2.1%clang@3.8.0-2ubuntu4 device=ch3 +hydra netmod=tcp +pmi+romio~verbs arch=linux-ubuntu16.04-x86_64 - ^findutils@4.6.0%clang@3.8.0-2ubuntu4 patches=84b916c0bf8c51b7e7b28417692f0ad3e7030d1f3c248ba77c42ede5c1c5d11e,bd9e4e5cc280f9753ae14956c4e4aa17fe7a210f55dd6c84aa60b12d106d47a2 arch=linux-ubuntu16.04-x86_64 - ^autoconf@2.69%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^m4@1.4.18%clang@3.8.0-2ubuntu4 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - ^libsigsegv@2.11%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^perl@5.26.2%clang@3.8.0-2ubuntu4+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac ~shared+threads arch=linux-ubuntu16.04-x86_64 - ^gdbm@1.14.1%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^readline@7.0%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^ncurses@6.1%clang@3.8.0-2ubuntu4~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - ^pkgconf@1.4.2%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^automake@1.16.1%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^libtool@2.4.6%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^texinfo@6.5%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^zlib@1.2.11%clang@3.8.0-2ubuntu4+optimize+pic~shared arch=linux-ubuntu16.04-x86_64 - - -So far we have only made global changes to the package preferences. As -we've seen throughout this tutorial, HDF5 builds with MPI enabled by -default in Spack. If we were working on a project that would routinely -need serial HDF5, that might get annoying quickly, having to type -``hdf5~mpi`` all the time. Instead, we'll update our preferences for -HDF5. - -.. code-block:: yaml - - packages: - all: - compiler: [clang, gcc, intel, pgi, xl, nag] - providers: - mpi: [mpich, openmpi] - variants: ~shared - hdf5: - variants: ~mpi - - -Now hdf5 will concretize without an MPI dependency by default. - -.. code-block:: console - - $ spack spec hdf5 - Input spec - -------------------------------- - hdf5 - - Concretized - -------------------------------- - hdf5@1.10.4%clang@3.8.0-2ubuntu4~cxx~debug~fortran~hl~mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - ^zlib@1.2.11%clang@3.8.0-2ubuntu4+optimize+pic~shared arch=linux-ubuntu16.04-x86_64 - - -In general, every attribute that we can set for all packages we can -set separately for an individual package. - -^^^^^^^^^^^^^^^^^ -External Packages -^^^^^^^^^^^^^^^^^ - -The packages configuration file also controls when Spack will build -against an externally installed package. On these systems we have a -pre-installed zlib. - -.. code-block:: yaml - - packages: - all: - compiler: [clang, gcc, intel, pgi, xl, nag] - providers: - mpi: [mpich, openmpi] - variants: ~shared - hdf5: - variants: ~mpi - zlib: - paths: - zlib@1.2.8%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64: /usr - - -Here, we've told Spack that zlib 1.2.8 is installed on our system. -We've also told it the installation prefix where zlib can be found. -We don't know exactly which variants it was built with, but that's -okay. - -.. code-block:: console - - $ spack spec hdf5 - Input spec - -------------------------------- - hdf5 - - Concretized - -------------------------------- - hdf5@1.10.4%gcc@5.4.0~cxx~debug~fortran~hl~mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - ^zlib@1.2.8%gcc@5.4.0+optimize+pic~shared arch=linux-ubuntu16.04-x86_64 - - -You'll notice that Spack is now using the external zlib installation, -but the compiler used to build zlib is now overriding our compiler -preference of clang. If we explicitly specify Clang: - -.. code-block:: console - - $ spack spec hdf5 %clang - Input spec - -------------------------------- - hdf5%clang - - Concretized - -------------------------------- - hdf5@1.10.4%clang@3.8.0-2ubuntu4~cxx~debug~fortran~hl~mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - ^zlib@1.2.11%clang@3.8.0-2ubuntu4+optimize+pic~shared arch=linux-ubuntu16.04-x86_64 - - -Spack concretizes to both HDF5 and zlib being built with Clang. -This has a side-effect of rebuilding zlib. If we want to force -Spack to use the system zlib, we have two choices. We can either -specify it on the command line, or we can tell Spack that it's -not allowed to build its own zlib. We'll go with the latter. - -.. code-block:: yaml - - packages: - all: - compiler: [clang, gcc, intel, pgi, xl, nag] - providers: - mpi: [mpich, openmpi] - variants: ~shared - hdf5: - variants: ~mpi - zlib: - paths: - zlib@1.2.8%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64: /usr - buildable: False - - -Now Spack will be forced to choose the external zlib. - -.. code-block:: console - - $ spack spec hdf5 %clang - Input spec - -------------------------------- - hdf5%clang - - Concretized - -------------------------------- - hdf5@1.10.4%clang@3.8.0-2ubuntu4~cxx~debug~fortran~hl~mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - ^zlib@1.2.8%gcc@5.4.0+optimize+pic~shared arch=linux-ubuntu16.04-x86_64 - - -This gets slightly more complicated with virtual dependencies. Suppose -we don't want to build our own MPI, but we now want a parallel version -of HDF5? Well, fortunately we have MPICH installed on these systems. - -.. code-block:: yaml - - packages: - all: - compiler: [clang, gcc, intel, pgi, xl, nag] - providers: - mpi: [mpich, openmpi] - variants: ~shared - hdf5: - variants: ~mpi - zlib: - paths: - zlib@1.2.8%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64: /usr - buildable: False - mpich: - paths: - mpich@3.2%gcc@5.4.0 device=ch3 +hydra netmod=tcp +pmi+romio~verbs arch=linux-ubuntu16.04-x86_64: /usr - buildable: False - - -If we concretize ``hdf5+mpi`` with this configuration file, we will just -build with an alternate MPI implementation. - -.. code-block:: console - - $ spack spec hdf5+mpi %clang - Input spec - -------------------------------- - hdf5%clang+mpi - - Concretized - -------------------------------- - hdf5@1.10.4%clang@3.8.0-2ubuntu4~cxx~debug~fortran~hl+mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - ^openmpi@3.1.3%clang@3.8.0-2ubuntu4~cuda+cxx_exceptions fabrics= ~java~legacylaunchers~memchecker~pmi schedulers= ~sqlite3~thread_multiple+vt arch=linux-ubuntu16.04-x86_64 - ^hwloc@1.11.9%clang@3.8.0-2ubuntu4~cairo~cuda+libxml2+pci~shared arch=linux-ubuntu16.04-x86_64 - ^libpciaccess@0.13.5%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^libtool@2.4.6%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^m4@1.4.18%clang@3.8.0-2ubuntu4 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - ^libsigsegv@2.11%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^pkgconf@1.4.2%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^util-macros@1.19.1%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^libxml2@2.9.8%clang@3.8.0-2ubuntu4~python arch=linux-ubuntu16.04-x86_64 - ^xz@5.2.4%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^zlib@1.2.8%gcc@5.4.0+optimize+pic~shared arch=linux-ubuntu16.04-x86_64 - ^numactl@2.0.11%clang@3.8.0-2ubuntu4 patches=592f30f7f5f757dfc239ad0ffd39a9a048487ad803c26b419e0f96b8cda08c1a arch=linux-ubuntu16.04-x86_64 - ^autoconf@2.69%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^perl@5.26.2%clang@3.8.0-2ubuntu4+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac ~shared+threads arch=linux-ubuntu16.04-x86_64 - ^gdbm@1.14.1%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^readline@7.0%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - ^ncurses@6.1%clang@3.8.0-2ubuntu4~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - ^automake@1.16.1%clang@3.8.0-2ubuntu4 arch=linux-ubuntu16.04-x86_64 - - -We have only expressed a preference for MPICH over other MPI -implementations, and Spack will happily build with one we haven't -forbid it from building. We could resolve this by requesting -``hdf5+mpi%clang^mpich`` explicitly, or we can configure Spack not to -use any other MPI implementation. Since we're focused on -configurations here and the former can get tedious, we'll need to -modify our ``packages.yaml`` file again. - -While we're at it, we can configure HDF5 to build with MPI by default -again. - -.. code-block:: yaml - - packages: - all: - compiler: [clang, gcc, intel, pgi, xl, nag] - providers: - mpi: [mpich, openmpi] - variants: ~shared - zlib: - paths: - zlib@1.2.8%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64: /usr - buildable: False - mpich: - paths: - mpich@3.2%gcc@5.4.0 device=ch3 +hydra netmod=tcp +pmi+romio~verbs arch=linux-ubuntu16.04-x86_64: /usr - buildable: False - openmpi: - buildable: False - mvapich2: - buildable: False - intel-mpi: - buildable: False - intel-parallel-studio: - buildable: False - spectrum-mpi: - buildable: False - mpilander: - buildable: False - charm: - buildable: False - charmpp: - buildable: False - - -Now that we have configured Spack not to build any of the possible -providers for MPI, we can try again. - -.. code-block:: console - - $ spack spec hdf5 %clang - Input spec - -------------------------------- - hdf5%clang - - Concretized - -------------------------------- - hdf5@1.10.4%clang@3.8.0-2ubuntu4~cxx~debug~fortran~hl+mpi+pic~shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - ^mpich@3.2%gcc@5.4.0 device=ch3 +hydra netmod=tcp +pmi+romio~verbs arch=linux-ubuntu16.04-x86_64 - ^zlib@1.2.8%gcc@5.4.0+optimize+pic~shared arch=linux-ubuntu16.04-x86_64 - - -By configuring most of our package preferences in ``packages.yaml``, -we can cut down on the amount of work we need to do when specifying -a spec on the command line. In addition to compiler and variant -preferences, we can specify version preferences as well. Except for -selecting providers via `^`, anything that you can specify on the -command line can be specified in ``packages.yaml`` with the exact -same spec syntax. - -^^^^^^^^^^^^^^^^^^^^^^^^ -Installation Permissions -^^^^^^^^^^^^^^^^^^^^^^^^ - -The ``packages.yaml`` file also controls the default permissions -to use when installing a package. You'll notice that by default, -the installation prefix will be world readable but only user writable. - -Let's say we need to install ``converge``, a licensed software package. -Since a specific research group, ``fluid_dynamics``, pays for this -license, we want to ensure that only members of this group can access -the software. We can do this like so: - -.. code-block:: yaml - - packages: - converge: - permissions: - read: group - group: fluid_dynamics - - -Now, only members of the ``fluid_dynamics`` group can use any -``converge`` installations. - -.. warning:: - - Make sure to delete or move the ``packages.yaml`` you have been - editing up to this point. Otherwise, it will change the hashes - of your packages, leading to differences in the output of later - tutorial sections. - - ------------------ -High-level Config ------------------ - -In addition to compiler and package settings, Spack allows customization -of several high-level settings. These settings are stored in the generic -``config.yaml`` configuration file. You can see the default settings by -running: - -.. code-block:: console - - $ spack config --scope defaults edit config - - -.. literalinclude:: _spack_root/etc/spack/defaults/config.yaml - :language: yaml - - -As you can see, many of the directories Spack uses can be customized. -For example, you can tell Spack to install packages to a prefix -outside of the ``$SPACK_ROOT`` hierarchy. Module files can be -written to a central location if you are using multiple Spack -instances. If you have a fast scratch file system, you can run builds -from this file system with the following ``config.yaml``: - -.. code-block:: yaml - - config: - build_stage: - - /scratch/$user/spack-stage - - -.. note:: - - It is important to distinguish the build stage directory from other - directories in your scratch space to ensure ``spack clean`` does not - inadvertently remove unrelated files. Spack prepends ``spack-stage-`` - to temporary staging directory names to reduce this risk. Using a - combination of ``spack`` and or ``stage`` in each specified path, as - shown in the default settings and documented examples, will add - another layer of protection. See :ref:`config-yaml` for details. - - -On systems with compilers that absolutely *require* environment variables -like ``LD_LIBRARY_PATH``, it is possible to prevent Spack from cleaning -the build environment with the ``dirty`` setting: - -.. code-block:: yaml - - config: - dirty: true - - -However, this is strongly discouraged, as it can pull unwanted libraries -into the build. - -One last setting that may be of interest to many users is the ability -to customize the parallelism of Spack builds. By default, Spack -installs all packages in parallel with the number of jobs equal to the -number of cores on the node (up to a maximum of 16). For example, on a -node with 16 cores, this will look like: - -.. code-block:: console - - $ spack install --no-cache --verbose --overwrite zlib - ==> Installing zlib - ==> Using cached archive: /home/user/spack/var/spack/cache/zlib/zlib-1.2.11.tar.gz - ==> Staging archive: /home/user/spack/var/spack/stage/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb/zlib-1.2.11.tar.gz - ==> Created stage in /home/user/spack/var/spack/stage/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - ==> No patches needed for zlib - ==> Building zlib [Package] - ==> Executing phase: 'install' - ==> './configure' '--prefix=/home/user/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb' - ... - ==> 'make' '-j16' - ... - ==> 'make' '-j16' 'install' - ... - ==> Successfully installed zlib - Fetch: 0.00s. Build: 1.03s. Total: 1.03s. - [+] /home/user/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - - -As you can see, we are building with all 16 cores on the node. If you are -on a shared login node, this can slow down the system for other users. If -you have a strict ulimit or restriction on the number of available licenses, -you may not be able to build at all with this many cores. On nodes with 64+ -cores, you may not see a significant speedup of the build anyway. To limit -the number of cores our build uses, set ``build_jobs`` like so: - -.. code-block:: yaml - - config: - build_jobs: 2 - - -If we uninstall and reinstall zlib, we see that it now uses only 2 cores: - -.. code-block:: console - - $ spack install --no-cache --verbose --overwrite zlib - ==> Installing zlib - ==> Using cached archive: /home/user/spack/var/spack/cache/zlib/zlib-1.2.11.tar.gz - ==> Staging archive: /home/user/spack/var/spack/stage/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb/zlib-1.2.11.tar.gz - ==> Created stage in /home/user/spack/var/spack/stage/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - ==> No patches needed for zlib - ==> Building zlib [Package] - ==> Executing phase: 'install' - ==> './configure' '--prefix=/home/user/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb' - ... - ==> 'make' '-j2' - ... - ==> 'make' '-j2' 'install' - ... - ==> Successfully installed zlib - Fetch: 0.00s. Build: 1.03s. Total: 1.03s. - [+] /home/user/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb - - -Obviously, if you want to build everything in serial for whatever reason, -you would set ``build_jobs`` to 1. - --------- -Examples --------- - -For examples of how other sites configure Spack, see -https://github.com/spack/spack-configs. If you use Spack at your site -and want to share your config files, feel free to submit a pull request! diff --git a/lib/spack/docs/tutorial_environments.rst b/lib/spack/docs/tutorial_environments.rst deleted file mode 100644 index 4b86dbd5ac..0000000000 --- a/lib/spack/docs/tutorial_environments.rst +++ /dev/null @@ -1,815 +0,0 @@ -.. Copyright 2013-2019 Lawrence Livermore National Security, LLC and other - Spack Project Developers. See the top-level COPYRIGHT file for details. - - SPDX-License-Identifier: (Apache-2.0 OR MIT) - -.. _environments-tutorial: - -===================== -Environments Tutorial -===================== - -We've shown you how to install and remove packages with Spack. You can -use :ref:`cmd-spack-install` to install packages, -:ref:`cmd-spack-uninstall` to remove them, and :ref:`cmd-spack-find` to -look at and query what is installed. We've also shown you how to -customize Spack's installation with configuration files like -:ref:`packages.yaml <build-settings>`. - -If you build a lot of software, or if you work on multiple projects, -managing everything in one place can be overwhelming. The default ``spack -find`` output may contain many packages, but you may want to *just* focus -on packages for a particular project. Moreover, you may want to include -special configuration with your package groups, e.g., to build all the -packages in the same group the same way. - -Spack **environments** provide a way to handle these problems. - -------------------- -Environment basics -------------------- - -Let's look at the output of ``spack find`` at this point in the tutorial. - -.. code-block:: console - - $ spack find - ==> 70 installed packages - -- linux-ubuntu16.04-x86_64 / clang@3.8.0-2ubuntu4 -------------- - tcl@8.6.8 zlib@1.2.8 zlib@1.2.11 - - -- linux-ubuntu16.04-x86_64 / gcc@4.7 --------------------------- - zlib@1.2.11 - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - adept-utils@1.0.1 hdf5@1.10.4 mpc@1.1.0 perl@5.26.2 - autoconf@2.69 hdf5@1.10.4 mpfr@3.1.6 pkgconf@1.4.2 - automake@1.16.1 hdf5@1.10.4 mpich@3.2.1 readline@7.0 - boost@1.68.0 hwloc@1.11.9 mpileaks@1.0 suite-sparse@5.3.0 - bzip2@1.0.6 hypre@2.15.1 mumps@5.1.1 tar@1.30 - callpath@1.0.4 hypre@2.15.1 mumps@5.1.1 tcl@8.6.8 - cmake@3.12.3 isl@0.18 ncurses@6.1 tcl@8.6.8 - diffutils@3.6 libdwarf@20180129 netcdf@4.6.1 texinfo@6.5 - dyninst@9.3.2 libiberty@2.31.1 netcdf@4.6.1 trilinos@12.12.1 - elfutils@0.173 libpciaccess@0.13.5 netlib-scalapack@2.0.2 trilinos@12.12.1 - findutils@4.6.0 libsigsegv@2.11 netlib-scalapack@2.0.2 util-macros@1.19.1 - gcc@7.2.0 libtool@2.4.6 numactl@2.0.11 xz@5.2.4 - gdbm@1.14.1 libxml2@2.9.8 openblas@0.3.3 zlib@1.2.8 - gettext@0.19.8.1 m4@1.4.18 openmpi@3.1.3 zlib@1.2.8 - glm@0.9.7.1 matio@1.5.9 openssl@1.0.2o zlib@1.2.11 - gmp@6.1.2 matio@1.5.9 parmetis@4.0.3 - hdf5@1.10.4 metis@5.1.0 parmetis@4.0.3 - - -This is a complete, but cluttered view. There are packages built with -both ``openmpi`` and ``mpich``, as well as multiple variants of other -packages, like ``zlib``. The query mechanism we learned about in ``spack -find`` can help, but it would be nice if we could start from a clean -slate without losing what we've already done. - - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Creating and activating environments -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The ``spack env`` command can help. Let's create a new environment: - -.. code-block:: console - - $ spack env create myproject - ==> Created environment 'myproject' in ~/spack/var/spack/environments/myproject - -An environment is a virtualized ``spack`` instance that you can use for a -specific purpose. You can see the environments we've created so far like this: - -.. code-block:: console - - $ spack env list - ==> 1 environments - myproject - -And you can **activate** an environment with ``spack env activate``: - -.. code-block:: console - - $ spack env activate myproject - -Once you enter an environment, ``spack find`` shows only what is in the -current environment. That's nothing, so far: - -.. code-block:: console - - $ spack find - ==> In environment myproject - ==> No root specs - - ==> 0 installed packages - -The ``spack find`` output is still *slightly* different. It tells you -that you're in the ``myproject`` environment, so that you don't panic -when you see that there is nothing installed. It also says that there -are *no root specs*. We'll get back to what that means later. - -If you *only* want to check what environment you are in, you can use -``spack env status``: - -.. code-block:: console - - $ spack env status - ==> In environment myproject - -And, if you want to leave this environment and go back to normal Spack, -you can use ``spack env deactivate``. We like to use the -``despacktivate`` alias (which Spack sets up automatically) for short: - -.. code-block:: console - - $ despacktivate # short alias for `spack env deactivate` - $ spack env status - ==> No active environment - $ spack find - netcdf@4.6.1 readline@7.0 zlib@1.2.11 - diffutils@3.6 hdf5@1.10.4 m4@1.4.18 netcdf@4.6.1 suite-sparse@5.3.0 - dyninst@10.0.0 hwloc@1.11.9 matio@1.5.9 netlib-scalapack@2.0.2 tar@1.30 - elfutils@0.173 hypre@2.15.1 matio@1.5.9 netlib-scalapack@2.0.2 tcl@8.6.8 - findutils@4.6.0 hypre@2.15.1 metis@5.1.0 numactl@2.0.11 tcl@8.6.8 - gcc@7.2.0 intel-tbb@2019 mpc@1.1.0 openblas@0.3.3 texinfo@6.5~ - - -^^^^^^^^^^^^^^^^^^^ -Installing packages -^^^^^^^^^^^^^^^^^^^ - -Ok, now that we understand how creation and activation work, let's go -back to ``myproject`` and *install* a few packages: - -.. code-block:: console - - $ spack env activate myproject - $ spack install tcl - ==> tcl is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/tcl-8.6.8-qhwyccywhx2i6s7ob2gvjrjtj3rnfuqt - $ spack install trilinos - ==> trilinos is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/trilinos-12.12.1-rlsruavxqvwk2tgxzxboclbo6ykjf54r - $ spack find - ==> In environment myproject - ==> Root specs - tcl trilinos - - ==> 22 installed packages - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - boost@1.68.0 hwloc@1.11.9 matio@1.5.9 netlib-scalapack@2.0.2 parmetis@4.0.3 xz@5.2.4 - bzip2@1.0.6 hypre@2.15.1 metis@5.1.0 numactl@2.0.11 suite-sparse@5.3.0 zlib@1.2.11 - glm@0.9.7.1 libpciaccess@0.13.5 mumps@5.1.1 openblas@0.3.3 tcl@8.6.8 - hdf5@1.10.4 libxml2@2.9.8 netcdf@4.6.1 openmpi@3.1.3 trilinos@12.12.1 - -We've installed ``tcl`` and ``trilinos`` in our environment, along with -all of their dependencies. We call ``tcl`` and ``trilinos`` the -**roots** because we asked for them explicitly. The other 20 packages -listed under "installed packages" are present because they were needed as -dependencies. So, these are the roots of the packages' dependency graph. - -The "<package> is already installed" messages above are generated because -we already installed these packages in previous steps of the tutorial, -and we don't have to rebuild them to put them in an environment. - -Now let's create *another* project. We'll call this one ``myproject2``: - -.. code-block:: console - - $ spack env create myproject2 - ==> Created environment 'myproject2' in ~/spack/var/spack/environments/myproject2 - $ spack env activate myproject2 - $ spack install hdf5 - ==> hdf5 is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4-ozyvmhzdew66byarohm4p36ep7wtcuiw - $ spack install trilinos - ==> trilinos is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/trilinos-12.12.1-rlsruavxqvwk2tgxzxboclbo6ykjf54r - $ spack find - ==> In environment myproject2 - ==> Root specs - hdf5 trilinos - - ==> 22 installed packages - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - boost@1.68.0 hdf5@1.10.4 libxml2@2.9.8 netcdf@4.6.1 openmpi@3.1.3 xz@5.2.4 - bzip2@1.0.6 hwloc@1.11.9 matio@1.5.9 netlib-scalapack@2.0.2 parmetis@4.0.3 zlib@1.2.11 - glm@0.9.7.1 hypre@2.15.1 metis@5.1.0 numactl@2.0.11 suite-sparse@5.3.0 - hdf5@1.10.4 libpciaccess@0.13.5 mumps@5.1.1 openblas@0.3.3 trilinos@12.12.1 - -Now we have two environments: one with ``tcl`` and ``trilinos``, and -another with ``hdf5`` and ``trilinos``. - -We can uninstall trilinos from ``myproject2`` as you would expect: - -.. code-block:: console - - $ spack uninstall trilinos - ==> The following packages will be uninstalled: - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - rlsruav trilinos@12.12.1%gcc~alloptpkgs+amesos+amesos2+anasazi+aztec+belos+boost build_type=RelWithDebInfo ~cgns~complex~dtk+epetra+epetraext+exodus+explicit_template_instantiation~float+fortran~fortrilinos+gtest+hdf5+hypre+ifpack+ifpack2~intrepid~intrepid2~isorropia+kokkos+metis~minitensor+ml+muelu+mumps~nox~openmp~phalanx~piro~pnetcdf~python~rol~rythmos+sacado~shards+shared~stk+suite-sparse~superlu~superlu-dist~teko~tempus+teuchos+tpetra~x11~xsdkflags~zlib+zoltan+zoltan2 - ==> Do you want to proceed? [y/N] y - $ spack find - ==> In environment myproject2 - ==> Root specs - hdf5 - - ==> 8 installed packages - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - hdf5@1.10.4 libpciaccess@0.13.5 numactl@2.0.11 xz@5.2.4 - hwloc@1.11.9 libxml2@2.9.8 openmpi@3.1.3 zlib@1.2.11 - -Now there is only one root spec, ``hdf5``, which requires fewer -additional dependencies. - -However, we still needed ``trilinos`` for the ``myproject`` environment! -What happened to it? Let's switch back and see. - -.. code-block:: console - - $ despacktivate - $ spack env activate myproject - $ spack find - ==> In environment myproject - ==> Root specs - tcl trilinos - - ==> 22 installed packages - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - boost@1.68.0 hwloc@1.11.9 matio@1.5.9 netlib-scalapack@2.0.2 parmetis@4.0.3 xz@5.2.4 - bzip2@1.0.6 hypre@2.15.1 metis@5.1.0 numactl@2.0.11 suite-sparse@5.3.0 zlib@1.2.11 - glm@0.9.7.1 libpciaccess@0.13.5 mumps@5.1.1 openblas@0.3.3 tcl@8.6.8 - hdf5@1.10.4 libxml2@2.9.8 netcdf@4.6.1 openmpi@3.1.3 trilinos@12.12.1 - - -Spack is smart enough to realize that ``trilinos`` is still present in -the other environment. Trilinos won't *actually* be uninstalled unless -it is no longer needed by any environments or packages. If it is still -needed, it is only removed from the environment. - -------------------------------- -Dealing with many specs at once -------------------------------- - -In the above examples, we just used ``install`` and ``uninstall``. There -are other ways to deal with groups of packages, as well. - -^^^^^^^^^^^^^ -Adding specs -^^^^^^^^^^^^^ - -Let's go back to our first ``myproject`` environment and *add* a few specs instead of installing them: - -.. code-block:: console - - $ spack add hdf5 - ==> Adding hdf5 to environment myproject - $ spack add gmp - ==> Adding mumps to environment myproject - $ spack find - ==> In environment myproject - ==> Root specs - gmp hdf5 tcl trilinos - - ==> 22 installed packages - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - boost@1.68.0 hwloc@1.11.9 matio@1.5.9 netlib-scalapack@2.0.2 parmetis@4.0.3 xz@5.2.4 - bzip2@1.0.6 hypre@2.15.1 metis@5.1.0 numactl@2.0.11 suite-sparse@5.3.0 zlib@1.2.11 - glm@0.9.7.1 libpciaccess@0.13.5 mumps@5.1.1 openblas@0.3.3 tcl@8.6.8 - hdf5@1.10.4 libxml2@2.9.8 netcdf@4.6.1 openmpi@3.1.3 trilinos@12.12.1 - -Let's take a close look at what happened. The two packages we added, -``hdf5`` and ``gmp``, are present, but they're not installed in the -environment yet. ``spack add`` just adds *roots* to the environment, but -it does not automatically install them. - -We can install *all* the as-yet uninstalled packages in an environment by -simply running ``spack install`` with no arguments: - -.. code-block:: console - - $ spack install - ==> Concretizing hdf5 - [+] ozyvmhz hdf5@1.10.4%gcc@5.4.0~cxx~debug~fortran~hl+mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - [+] 3njc4q5 ^openmpi@3.1.3%gcc@5.4.0~cuda+cxx_exceptions fabrics= ~java~legacylaunchers~memchecker~pmi schedulers= ~sqlite3~thread_multiple+vt arch=linux-ubuntu16.04-x86_64 - [+] 43tkw5m ^hwloc@1.11.9%gcc@5.4.0~cairo~cuda+libxml2+pci+shared arch=linux-ubuntu16.04-x86_64 - [+] 5urc6tc ^libpciaccess@0.13.5%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] o2pfwjf ^libtool@2.4.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] suf5jtc ^m4@1.4.18%gcc@5.4.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - [+] fypapcp ^libsigsegv@2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] fovrh7a ^pkgconf@1.4.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] milz7fm ^util-macros@1.19.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] wpexsph ^libxml2@2.9.8%gcc@5.4.0~python arch=linux-ubuntu16.04-x86_64 - [+] teneqii ^xz@5.2.4%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 5nus6kn ^zlib@1.2.11%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 - [+] ft463od ^numactl@2.0.11%gcc@5.4.0 patches=592f30f7f5f757dfc239ad0ffd39a9a048487ad803c26b419e0f96b8cda08c1a arch=linux-ubuntu16.04-x86_64 - [+] 3sx2gxe ^autoconf@2.69%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] ic2kyoa ^perl@5.26.2%gcc@5.4.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - [+] q4fpyuo ^gdbm@1.14.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] nxhwrg7 ^readline@7.0%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 3o765ou ^ncurses@6.1%gcc@5.4.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - [+] rymw7im ^automake@1.16.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ==> Concretizing gmp - [+] qc4qcfz gmp@6.1.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 3sx2gxe ^autoconf@2.69%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] suf5jtc ^m4@1.4.18%gcc@5.4.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - [+] fypapcp ^libsigsegv@2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] ic2kyoa ^perl@5.26.2%gcc@5.4.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - [+] q4fpyuo ^gdbm@1.14.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] nxhwrg7 ^readline@7.0%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 3o765ou ^ncurses@6.1%gcc@5.4.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - [+] fovrh7a ^pkgconf@1.4.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] rymw7im ^automake@1.16.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] o2pfwjf ^libtool@2.4.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ==> Installing environment myproject - ==> tcl is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/tcl-8.6.8-qhwyccywhx2i6s7ob2gvjrjtj3rnfuqt - ==> trilinos is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/trilinos-12.12.1-rlsruavxqvwk2tgxzxboclbo6ykjf54r - ==> hdf5 is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/hdf5-1.10.4-ozyvmhzdew66byarohm4p36ep7wtcuiw - ==> gmp is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gmp-6.1.2-qc4qcfz4monpllc3nqupdo7vwinf73sw - -Spack will concretize the new roots, and install everything you added to -the environment. Now we can see the installed roots in the output of -``spack find``: - -.. code-block:: console - - $ spack find - ==> In environment myproject - ==> Root specs - gmp hdf5 tcl trilinos - - ==> 24 installed packages - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - boost@1.68.0 hdf5@1.10.4 libpciaccess@0.13.5 mumps@5.1.1 openblas@0.3.3 tcl@8.6.8 - bzip2@1.0.6 hdf5@1.10.4 libxml2@2.9.8 netcdf@4.6.1 openmpi@3.1.3 trilinos@12.12.1 - glm@0.9.7.1 hwloc@1.11.9 matio@1.5.9 netlib-scalapack@2.0.2 parmetis@4.0.3 xz@5.2.4 - gmp@6.1.2 hypre@2.15.1 metis@5.1.0 numactl@2.0.11 suite-sparse@5.3.0 zlib@1.2.11 - -We can build whole environments this way, by adding specs and installing -all at once, or we can install them with the usual ``install`` and -``uninstall`` portions. The advantage to doing them all at once is that -we don't have to write a script outside of Spack to automate this, and we -can kick off a large build of many packages easily. - -^^^^^^^^^^^^^^^^^^^^^ -Configuration -^^^^^^^^^^^^^^^^^^^^^ - -So far, ``myproject`` does not have any special configuration associated -with it. The specs concretize using Spack's defaults: - -.. code-block:: console - - $ spack spec hypre - Input spec - -------------------------------- - hypre - - Concretized - -------------------------------- - hypre@2.15.1%gcc@5.4.0~debug~int64+internal-superlu+mpi+shared arch=linux-ubuntu16.04-x86_64 - ^openblas@0.3.3%gcc@5.4.0 cpu_target= ~ilp64 patches=47cfa7a952ac7b2e4632c73ae199d69fb54490627b66a62c681e21019c4ddc9d,714aea33692304a50bd0ccde42590c176c82ded4a8ac7f06e573dc8071929c33 +pic+shared threads=none ~virtual_machine arch=linux-ubuntu16.04-x86_64 - ^openmpi@3.1.3%gcc@5.4.0~cuda+cxx_exceptions fabrics= ~java~legacylaunchers~memchecker~pmi schedulers= ~sqlite3~thread_multiple+vt arch=linux-ubuntu16.04-x86_64 - ^hwloc@1.11.9%gcc@5.4.0~cairo~cuda+libxml2+pci+shared arch=linux-ubuntu16.04-x86_64 - ^libpciaccess@0.13.5%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^libtool@2.4.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^m4@1.4.18%gcc@5.4.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - ^libsigsegv@2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^pkgconf@1.4.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^util-macros@1.19.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^libxml2@2.9.8%gcc@5.4.0~python arch=linux-ubuntu16.04-x86_64 - ^xz@5.2.4%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^zlib@1.2.11%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 - ^numactl@2.0.11%gcc@5.4.0 patches=592f30f7f5f757dfc239ad0ffd39a9a048487ad803c26b419e0f96b8cda08c1a arch=linux-ubuntu16.04-x86_64 - ^autoconf@2.69%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^perl@5.26.2%gcc@5.4.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - ^gdbm@1.14.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^readline@7.0%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^ncurses@6.1%gcc@5.4.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - ^automake@1.16.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - -You may want to add extra configuration to your environment. You can see -how your environment is configured using ``spack config get``: - -.. code-block:: console - - $ spack config get - # This is a Spack Environment file. - # - # It describes a set of packages to be installed, along with - # configuration settings. - spack: - # add package specs to the `specs` list - specs: [tcl, trilinos, hdf5, gmp] - -It turns out that this is a special configuration format where Spack -stores the state for the environment. Currently, the file is just a -``spack:`` header and a list of ``specs``. These are the roots. - -You can edit this file to add your own custom configuration. Spack -provides a shortcut to do that: - -.. code-block:: console - - spack config edit - -You should now see the same file, and edit it to look like this: - -.. code-block:: yaml - - # This is a Spack Environment file. - # - # It describes a set of packages to be installed, along with - # configuration settings. - spack: - packages: - all: - providers: - mpi: [mpich] - - # add package specs to the `specs` list - specs: [tcl, trilinos, hdf5, gmp] - -Now if we run ``spack spec`` again in the environment, specs will concretize with ``mpich`` as the MPI implementation: - -.. code-block:: console - - $ spack spec hypre - Input spec - -------------------------------- - hypre - - Concretized - -------------------------------- - hypre@2.15.1%gcc@5.4.0~debug~int64+internal-superlu+mpi+shared arch=linux-ubuntu16.04-x86_64 - ^mpich@3.2.1%gcc@5.4.0 device=ch3 +hydra netmod=tcp +pmi+romio~verbs arch=linux-ubuntu16.04-x86_64 - ^findutils@4.6.0%gcc@5.4.0 patches=84b916c0bf8c51b7e7b28417692f0ad3e7030d1f3c248ba77c42ede5c1c5d11e,bd9e4e5cc280f9753ae14956c4e4aa17fe7a210f55dd6c84aa60b12d106d47a2 arch=linux-ubuntu16.04-x86_64 - ^autoconf@2.69%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^m4@1.4.18%gcc@5.4.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - ^libsigsegv@2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^perl@5.26.2%gcc@5.4.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - ^gdbm@1.14.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^readline@7.0%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^ncurses@6.1%gcc@5.4.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - ^pkgconf@1.4.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^automake@1.16.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^libtool@2.4.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^texinfo@6.5%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ^openblas@0.3.3%gcc@5.4.0 cpu_target= ~ilp64 patches=47cfa7a952ac7b2e4632c73ae199d69fb54490627b66a62c681e21019c4ddc9d,714aea33692304a50bd0ccde42590c176c82ded4a8ac7f06e573dc8071929c33 +pic+shared threads=none ~virtual_machine arch=linux-ubuntu16.04-x86_64 - -In addition to the ``specs`` section, an environment's configuration can -contain any of the configuration options from Spack's various config -sections. You can add custom repositories, a custom install location, -custom compilers, or custom external packages, in addition to the ``package`` -preferences we show here. - -But now we have a problem. We already installed part of this environment -with openmpi, but now we want to install it with ``mpich``. - -You can run ``spack concretize`` inside of an environment to concretize -all of its specs. We can run it here: - -.. code-block:: console - - $ spack concretize -f - ==> Concretizing tcl - [+] qhwyccy tcl@8.6.8%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 5nus6kn ^zlib@1.2.11%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 - ==> Concretizing trilinos - [+] kqc52mo trilinos@12.12.1%gcc@5.4.0~alloptpkgs+amesos+amesos2+anasazi+aztec+belos+boost build_type=RelWithDebInfo ~cgns~complex~dtk+epetra+epetraext+exodus+explicit_template_instantiation~float+fortran~fortrilinos+gtest+hdf5+hypre+ifpack+ifpack2~intrepid~intrepid2~isorropia+kokkos+metis~minitensor+ml+muelu+mumps~nox~openmp~phalanx~piro~pnetcdf~python~rol~rythmos+sacado~shards+shared~stk+suite-sparse~superlu~superlu-dist~teko~tempus+teuchos+tpetra~x11~xsdkflags~zlib+zoltan+zoltan2 arch=linux-ubuntu16.04-x86_64 - [+] zbgfxap ^boost@1.68.0%gcc@5.4.0+atomic+chrono~clanglibcpp cxxstd=default +date_time~debug+exception+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy patches=2ab6c72d03dec6a4ae20220a9dfd5c8c572c5294252155b85c6874d97c323199 +program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave arch=linux-ubuntu16.04-x86_64 - [+] ufczdvs ^bzip2@1.0.6%gcc@5.4.0+shared arch=linux-ubuntu16.04-x86_64 - [+] 2rhuivg ^diffutils@3.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 5nus6kn ^zlib@1.2.11%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 - [+] otafqzh ^cmake@3.12.3%gcc@5.4.0~doc+ncurses+openssl+ownlibs patches=dd3a40d4d92f6b2158b87d6fb354c277947c776424aa03f6dc8096cf3135f5d0 ~qt arch=linux-ubuntu16.04-x86_64 - [+] 3o765ou ^ncurses@6.1%gcc@5.4.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - [+] fovrh7a ^pkgconf@1.4.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] b4y3w3b ^openssl@1.0.2o%gcc@5.4.0+systemcerts arch=linux-ubuntu16.04-x86_64 - [+] ic2kyoa ^perl@5.26.2%gcc@5.4.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - [+] q4fpyuo ^gdbm@1.14.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] nxhwrg7 ^readline@7.0%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] jnw622j ^glm@0.9.7.1%gcc@5.4.0 build_type=RelWithDebInfo arch=linux-ubuntu16.04-x86_64 - [+] xxd7syh ^hdf5@1.10.4%gcc@5.4.0~cxx~debug~fortran+hl+mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - [+] p3f7p2r ^mpich@3.2.1%gcc@5.4.0 device=ch3 +hydra netmod=tcp +pmi+romio~verbs arch=linux-ubuntu16.04-x86_64 - [+] d4iajxs ^findutils@4.6.0%gcc@5.4.0 patches=84b916c0bf8c51b7e7b28417692f0ad3e7030d1f3c248ba77c42ede5c1c5d11e,bd9e4e5cc280f9753ae14956c4e4aa17fe7a210f55dd6c84aa60b12d106d47a2 arch=linux-ubuntu16.04-x86_64 - [+] 3sx2gxe ^autoconf@2.69%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] suf5jtc ^m4@1.4.18%gcc@5.4.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - [+] fypapcp ^libsigsegv@2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] rymw7im ^automake@1.16.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] o2pfwjf ^libtool@2.4.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] zs7a2pc ^texinfo@6.5%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] obewuoz ^hypre@2.15.1%gcc@5.4.0~debug~int64~internal-superlu+mpi+shared arch=linux-ubuntu16.04-x86_64 - [+] cyeg2yi ^openblas@0.3.3%gcc@5.4.0 cpu_target= ~ilp64 patches=47cfa7a952ac7b2e4632c73ae199d69fb54490627b66a62c681e21019c4ddc9d,714aea33692304a50bd0ccde42590c176c82ded4a8ac7f06e573dc8071929c33 +pic+shared threads=none ~virtual_machine arch=linux-ubuntu16.04-x86_64 - [+] gvyqldh ^matio@1.5.9%gcc@5.4.0+hdf5+shared+zlib arch=linux-ubuntu16.04-x86_64 - [+] 3wnvp4j ^metis@5.1.0%gcc@5.4.0 build_type=Release ~gdb~int64 patches=4991da938c1d3a1d3dea78e49bbebecba00273f98df2a656e38b83d55b281da1 ~real64+shared arch=linux-ubuntu16.04-x86_64 - [+] cumcj5a ^mumps@5.1.1%gcc@5.4.0+complex+double+float~int64~metis+mpi~parmetis~ptscotch~scotch+shared arch=linux-ubuntu16.04-x86_64 - [+] p7iln2p ^netlib-scalapack@2.0.2%gcc@5.4.0 build_type=RelWithDebInfo ~pic+shared arch=linux-ubuntu16.04-x86_64 - [+] wmmx5sg ^netcdf@4.6.1%gcc@5.4.0~dap~hdf4 maxdims=1024 maxvars=8192 +mpi~parallel-netcdf+shared arch=linux-ubuntu16.04-x86_64 - [+] jehtata ^parmetis@4.0.3%gcc@5.4.0 build_type=RelWithDebInfo ~gdb patches=4f892531eb0a807eb1b82e683a416d3e35154a455274cf9b162fb02054d11a5b,50ed2081bc939269689789942067c58b3e522c269269a430d5d34c00edbc5870,704b84f7c7444d4372cb59cca6e1209df4ef3b033bc4ee3cf50f369bce972a9d +shared arch=linux-ubuntu16.04-x86_64 - [+] zaau4ki ^suite-sparse@5.3.0%gcc@5.4.0~cuda~openmp+pic~tbb arch=linux-ubuntu16.04-x86_64 - ==> Concretizing hdf5 - - zjgyn3w hdf5@1.10.4%gcc@5.4.0~cxx~debug~fortran~hl+mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - [+] p3f7p2r ^mpich@3.2.1%gcc@5.4.0 device=ch3 +hydra netmod=tcp +pmi+romio~verbs arch=linux-ubuntu16.04-x86_64 - [+] d4iajxs ^findutils@4.6.0%gcc@5.4.0 patches=84b916c0bf8c51b7e7b28417692f0ad3e7030d1f3c248ba77c42ede5c1c5d11e,bd9e4e5cc280f9753ae14956c4e4aa17fe7a210f55dd6c84aa60b12d106d47a2 arch=linux-ubuntu16.04-x86_64 - [+] 3sx2gxe ^autoconf@2.69%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] suf5jtc ^m4@1.4.18%gcc@5.4.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - [+] fypapcp ^libsigsegv@2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] ic2kyoa ^perl@5.26.2%gcc@5.4.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - [+] q4fpyuo ^gdbm@1.14.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] nxhwrg7 ^readline@7.0%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 3o765ou ^ncurses@6.1%gcc@5.4.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - [+] fovrh7a ^pkgconf@1.4.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] rymw7im ^automake@1.16.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] o2pfwjf ^libtool@2.4.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] zs7a2pc ^texinfo@6.5%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 5nus6kn ^zlib@1.2.11%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 - ==> Concretizing gmp - [+] qc4qcfz gmp@6.1.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 3sx2gxe ^autoconf@2.69%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] suf5jtc ^m4@1.4.18%gcc@5.4.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - [+] fypapcp ^libsigsegv@2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] ic2kyoa ^perl@5.26.2%gcc@5.4.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - [+] q4fpyuo ^gdbm@1.14.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] nxhwrg7 ^readline@7.0%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 3o765ou ^ncurses@6.1%gcc@5.4.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - [+] fovrh7a ^pkgconf@1.4.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] rymw7im ^automake@1.16.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] o2pfwjf ^libtool@2.4.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - -Now, all the specs in the environment are concrete and ready to be -installed with ``mpich`` as the MPI implementation. - -Normally, we could just run ``spack config edit``, edit the environment -configuration, ``spack add`` some specs, and ``spack install``. - -But, when we already have installed packages in the environment, we have -to force everything in the environment to be re-concretized using ``spack -concretize -f``. *Then* we can re-run ``spack install``. - - ------------------------------------ -``spack.yaml`` and ``spack.lock`` ------------------------------------ - -So far we've shown you how to interact with environments from the command -line, but they also have a file-based interface that can be used by -developers and admins to manage workflows for projects. - -In this section we'll dive a little deeper to see how environments are -implemented, and how you could use this in your day-to-day development. - -^^^^^^^^^^^^^^ -``spack.yaml`` -^^^^^^^^^^^^^^ - -Earlier, we changed an environment's configuration using ``spack config -edit``. We were actually editing a special file called ``spack.yaml``. -Let's take a look. - -We can get directly to the current environment's location using ``spack cd``: - -.. code-block:: console - - $ spack cd -e myproject - $ pwd - ~/spack/var/spack/environments/myproject - $ ls - spack.lock spack.yaml - -We notice two things here. First, the environment is just a directory -inside of ``var/spack/environments`` within the Spack installation. -Second, it contains two important files: ``spack.yaml`` and -``spack.lock``. - -``spack.yaml`` is the configuration file for environments that we've -already seen, but it does not *have* to live inside Spack. If you create -an environment using ``spack env create``, it is *managed* by -Spack in the ``var/spack/environments`` directory, and you can refer to -it by name. - -You can actually put a ``spack.yaml`` file *anywhere*, and you can use it -to bundle an environment, or a list of dependencies to install, with your -project. Let's make a simple project: - -.. code-block:: console - - $ cd - $ mkdir code - $ cd code - $ spack env create -d . - ==> Created environment in ~/code - -Here, we made a new directory called *code*, and we used the ``-d`` -option to create an environment in it. - -What really happened? - -.. code-block:: console - - $ ls - spack.yaml - $ cat spack.yaml - # This is a Spack Environment file. - # - # It describes a set of packages to be installed, along with - # configuration settings. - spack: - # add package specs to the `specs` list - specs: [] - -Spack just created a ``spack.yaml`` file in the code directory, with an -empty list of root specs. Now we have a Spack environment, *in a -directory*, that we can use to manage dependencies. Suppose your project -depends on ``boost``, ``trilinos``, and ``openmpi``. You can add these -to your spec list: - -.. code-block:: yaml - - # This is a Spack Environment file. - # - # It describes a set of packages to be installed, along with - # configuration settings. - spack: - # add package specs to the `specs` list - specs: - - boost - - trilinos - - openmpi - -And now *anyone* who uses the *code* repository can use this format to -install the project's dependencies. They need only clone the repository, -``cd`` into it, and type ``spack install``: - -.. code-block:: console - - $ spack install - ==> Concretizing boost - [+] zbgfxap boost@1.68.0%gcc@5.4.0+atomic+chrono~clanglibcpp cxxstd=default +date_time~debug+exception+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy patches=2ab6c72d03dec6a4ae20220a9dfd5c8c572c5294252155b85c6874d97c323199 +program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave arch=linux-ubuntu16.04-x86_64 - [+] ufczdvs ^bzip2@1.0.6%gcc@5.4.0+shared arch=linux-ubuntu16.04-x86_64 - [+] 2rhuivg ^diffutils@3.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 5nus6kn ^zlib@1.2.11%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 - ==> Concretizing trilinos - [+] rlsruav trilinos@12.12.1%gcc@5.4.0~alloptpkgs+amesos+amesos2+anasazi+aztec+belos+boost build_type=RelWithDebInfo ~cgns~complex~dtk+epetra+epetraext+exodus+explicit_template_instantiation~float+fortran~fortrilinos+gtest+hdf5+hypre+ifpack+ifpack2~intrepid~intrepid2~isorropia+kokkos+metis~minitensor+ml+muelu+mumps~nox~openmp~phalanx~piro~pnetcdf~python~rol~rythmos+sacado~shards+shared~stk+suite-sparse~superlu~superlu-dist~teko~tempus+teuchos+tpetra~x11~xsdkflags~zlib+zoltan+zoltan2 arch=linux-ubuntu16.04-x86_64 - [+] zbgfxap ^boost@1.68.0%gcc@5.4.0+atomic+chrono~clanglibcpp cxxstd=default +date_time~debug+exception+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy patches=2ab6c72d03dec6a4ae20220a9dfd5c8c572c5294252155b85c6874d97c323199 +program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave arch=linux-ubuntu16.04-x86_64 - [+] ufczdvs ^bzip2@1.0.6%gcc@5.4.0+shared arch=linux-ubuntu16.04-x86_64 - [+] 2rhuivg ^diffutils@3.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 5nus6kn ^zlib@1.2.11%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 - [+] otafqzh ^cmake@3.12.3%gcc@5.4.0~doc+ncurses+openssl+ownlibs patches=dd3a40d4d92f6b2158b87d6fb354c277947c776424aa03f6dc8096cf3135f5d0 ~qt arch=linux-ubuntu16.04-x86_64 - [+] 3o765ou ^ncurses@6.1%gcc@5.4.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - [+] fovrh7a ^pkgconf@1.4.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] b4y3w3b ^openssl@1.0.2o%gcc@5.4.0+systemcerts arch=linux-ubuntu16.04-x86_64 - [+] ic2kyoa ^perl@5.26.2%gcc@5.4.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - [+] q4fpyuo ^gdbm@1.14.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] nxhwrg7 ^readline@7.0%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] jnw622j ^glm@0.9.7.1%gcc@5.4.0 build_type=RelWithDebInfo arch=linux-ubuntu16.04-x86_64 - [+] oqwnui7 ^hdf5@1.10.4%gcc@5.4.0~cxx~debug~fortran+hl+mpi+pic+shared~szip~threadsafe arch=linux-ubuntu16.04-x86_64 - [+] 3njc4q5 ^openmpi@3.1.3%gcc@5.4.0~cuda+cxx_exceptions fabrics= ~java~legacylaunchers~memchecker~pmi schedulers= ~sqlite3~thread_multiple+vt arch=linux-ubuntu16.04-x86_64 - [+] 43tkw5m ^hwloc@1.11.9%gcc@5.4.0~cairo~cuda+libxml2+pci+shared arch=linux-ubuntu16.04-x86_64 - [+] 5urc6tc ^libpciaccess@0.13.5%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] o2pfwjf ^libtool@2.4.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] suf5jtc ^m4@1.4.18%gcc@5.4.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - [+] fypapcp ^libsigsegv@2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] milz7fm ^util-macros@1.19.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] wpexsph ^libxml2@2.9.8%gcc@5.4.0~python arch=linux-ubuntu16.04-x86_64 - [+] teneqii ^xz@5.2.4%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] ft463od ^numactl@2.0.11%gcc@5.4.0 patches=592f30f7f5f757dfc239ad0ffd39a9a048487ad803c26b419e0f96b8cda08c1a arch=linux-ubuntu16.04-x86_64 - [+] 3sx2gxe ^autoconf@2.69%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] rymw7im ^automake@1.16.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] fshksdp ^hypre@2.15.1%gcc@5.4.0~debug~int64~internal-superlu+mpi+shared arch=linux-ubuntu16.04-x86_64 - [+] cyeg2yi ^openblas@0.3.3%gcc@5.4.0 cpu_target= ~ilp64 patches=47cfa7a952ac7b2e4632c73ae199d69fb54490627b66a62c681e21019c4ddc9d,714aea33692304a50bd0ccde42590c176c82ded4a8ac7f06e573dc8071929c33 +pic+shared threads=none ~virtual_machine arch=linux-ubuntu16.04-x86_64 - [+] lmzdgss ^matio@1.5.9%gcc@5.4.0+hdf5+shared+zlib arch=linux-ubuntu16.04-x86_64 - [+] 3wnvp4j ^metis@5.1.0%gcc@5.4.0 build_type=Release ~gdb~int64 patches=4991da938c1d3a1d3dea78e49bbebecba00273f98df2a656e38b83d55b281da1 ~real64+shared arch=linux-ubuntu16.04-x86_64 - [+] acsg2dz ^mumps@5.1.1%gcc@5.4.0+complex+double+float~int64~metis+mpi~parmetis~ptscotch~scotch+shared arch=linux-ubuntu16.04-x86_64 - [+] wotpfwf ^netlib-scalapack@2.0.2%gcc@5.4.0 build_type=RelWithDebInfo ~pic+shared arch=linux-ubuntu16.04-x86_64 - [+] mhm4izp ^netcdf@4.6.1%gcc@5.4.0~dap~hdf4 maxdims=1024 maxvars=8192 +mpi~parallel-netcdf+shared arch=linux-ubuntu16.04-x86_64 - [+] uv6h3sq ^parmetis@4.0.3%gcc@5.4.0 build_type=RelWithDebInfo ~gdb patches=4f892531eb0a807eb1b82e683a416d3e35154a455274cf9b162fb02054d11a5b,50ed2081bc939269689789942067c58b3e522c269269a430d5d34c00edbc5870,704b84f7c7444d4372cb59cca6e1209df4ef3b033bc4ee3cf50f369bce972a9d +shared arch=linux-ubuntu16.04-x86_64 - [+] zaau4ki ^suite-sparse@5.3.0%gcc@5.4.0~cuda~openmp+pic~tbb arch=linux-ubuntu16.04-x86_64 - ==> Concretizing openmpi - [+] 3njc4q5 openmpi@3.1.3%gcc@5.4.0~cuda+cxx_exceptions fabrics= ~java~legacylaunchers~memchecker~pmi schedulers= ~sqlite3~thread_multiple+vt arch=linux-ubuntu16.04-x86_64 - [+] 43tkw5m ^hwloc@1.11.9%gcc@5.4.0~cairo~cuda+libxml2+pci+shared arch=linux-ubuntu16.04-x86_64 - [+] 5urc6tc ^libpciaccess@0.13.5%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] o2pfwjf ^libtool@2.4.6%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] suf5jtc ^m4@1.4.18%gcc@5.4.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu16.04-x86_64 - [+] fypapcp ^libsigsegv@2.11%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] fovrh7a ^pkgconf@1.4.2%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] milz7fm ^util-macros@1.19.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] wpexsph ^libxml2@2.9.8%gcc@5.4.0~python arch=linux-ubuntu16.04-x86_64 - [+] teneqii ^xz@5.2.4%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 5nus6kn ^zlib@1.2.11%gcc@5.4.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64 - [+] ft463od ^numactl@2.0.11%gcc@5.4.0 patches=592f30f7f5f757dfc239ad0ffd39a9a048487ad803c26b419e0f96b8cda08c1a arch=linux-ubuntu16.04-x86_64 - [+] 3sx2gxe ^autoconf@2.69%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] ic2kyoa ^perl@5.26.2%gcc@5.4.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-ubuntu16.04-x86_64 - [+] q4fpyuo ^gdbm@1.14.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] nxhwrg7 ^readline@7.0%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - [+] 3o765ou ^ncurses@6.1%gcc@5.4.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64 - [+] rymw7im ^automake@1.16.1%gcc@5.4.0 arch=linux-ubuntu16.04-x86_64 - ==> Installing environment ~/code - ==> boost is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/boost-1.68.0-zbgfxapchxa4awxdwpleubfuznblxzvt - ==> trilinos is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/trilinos-12.12.1-rlsruavxqvwk2tgxzxboclbo6ykjf54r - ==> openmpi is already installed in ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.1.3-3njc4q5pqdpptq6jvqjrezkffwokv2sx - - -Spack concretizes the specs in the ``spack.yaml`` file and installs them. - -What happened here? If you ``cd`` into a directory that has a -``spack.yaml`` file in it, Spack considers this directory's environment -to be activated. The directory does not have to live within Spack; it -can be anywhere. - -So, from ``~/code``, we can actually manipulate ``spack.yaml`` using -``spack add`` and ``spack remove`` (just like managed environments): - -.. code-block:: console - - $ spack add hdf5@5.5.1 - ==> Adding hdf5 to environment ~/code - $ cat spack.yaml - # This is a Spack Environment file. - # - # It describes a set of packages to be installed, along with - # configuration settings. - spack: - # add package specs to the `specs` list - specs: - - boost - - trilinos - - openmpi - - hdf5@5.5.1 - - $ spack remove hdf5 - ==> Removing hdf5 from environment ~/code - $ cat spack.yaml - # This is a Spack Environment file. - # - # It describes a set of packages to be installed, along with - # configuration settings. - spack: - # add package specs to the `specs` list - specs: - - boost - - trilinos - - openmpi - - -^^^^^^^^^^^^^^ -``spack.lock`` -^^^^^^^^^^^^^^ - -Okay, we've covered managed environments, environments in directories, and -the last thing we'll cover is ``spack.lock``. You may remember that when -we ran ``spack install``, Spack concretized all the specs in the -``spack.yaml`` file and installed them. - -Whenever we concretize Specs in an environment, all concrete specs in the -environment are written out to a ``spack.lock`` file *alongside* -``spack.yaml``. The ``spack.lock`` file is not really human-readable -like the ``spack.yaml`` file. It is a ``json`` format that contains all -the information that we need to *reproduce* the build of an -environment: - -.. code-block:: console - - $ head spack.lock - { - "concrete_specs": { - "teneqii2xv5u6zl5r6qi3pwurc6pmypz": { - "xz": { - "version": "5.2.4", - "arch": { - "platform": "linux", - "platform_os": "ubuntu16.04", - "target": "x86_64" - }, - ... - -``spack.yaml`` and ``spack.lock`` correspond to two fundamental concepts -in Spack, but for environments: - - * ``spack.yaml`` is the set of *abstract* specs and configuration that - you want to install. - * ``spack.lock`` is the set of all fully *concretized* specs generated - from concretizing ``spack.yaml`` - -Using either of these, you can recreate an environment that someone else -built. ``spack env create`` takes an extra optional argument, which can -be either a ``spack.yaml`` or a ``spack.lock`` file: - -.. code-block:: console - - $ spack env create my-project spack.yaml - - $ spack env create my-project spack.lock - -Both of these create a new environment called ``my-project``, but which -one you choose to use depends on your needs: - -#. copying the yaml file allows someone else to build your *requirements*, - potentially a different way. - -#. copying the lock file allows someone else to rebuild your - *installation* exactly as you built it. - -The first use case can *re-concretize* the same specs on new platforms in -order to build, but it will preserve the abstract requirements. The -second use case (currently) requires you to be on the same machine, but -it retains all decisions made during concretization and is faithful to a -prior install. diff --git a/lib/spack/docs/tutorial_modules.rst b/lib/spack/docs/tutorial_modules.rst deleted file mode 100644 index 686da9f26e..0000000000 --- a/lib/spack/docs/tutorial_modules.rst +++ /dev/null @@ -1,1575 +0,0 @@ -.. Copyright 2013-2019 Lawrence Livermore National Security, LLC and other - Spack Project Developers. See the top-level COPYRIGHT file for details. - - SPDX-License-Identifier: (Apache-2.0 OR MIT) - -.. _modules-tutorial: - -============ -Module Files -============ - -In this tutorial, we'll introduce a few concepts that are fundamental -to the generation of module files with Spack, and we'll guide you through -the customization of both module files content and their layout on disk. In the end you -should have a clear understanding of: - - * What are module files and how they work - * How Spack generates them - * Which commands are available to ease their maintenance - * How it is possible to customize them in all aspects - -.. _module_file_tutorial_overview: - -------------------- -Modules at a glance -------------------- - -Let's start by summarizing what module files are and how you can use -them to modify your environment. The idea is to give enough information so that -people without any previous exposure to them will be able to follow the tutorial -later on. We'll also give a high-level view of how module files are generated -in Spack. If you are already familiar with these topics you can quickly skim -through this section or move directly to :ref:`module_file_tutorial_prerequisites`. - -.. _module_file_tutorial_what_are_modules: - -^^^^^^^^^^^^^^^^^^^^^^ -What are module files? -^^^^^^^^^^^^^^^^^^^^^^ - -Module files are an easy way to modify your environment in a controlled manner -during a shell session. In general, they contain the information needed to run an -application or use a library, and they work in conjunction with a tool that -interprets them. -Typical module files instruct this tool to modify the environment variables when a -module file is loaded: - - .. code-block:: console - - $ module show zlib - ------------------------------------------------------------------- - /home/mculpo/PycharmProjects/spack/share/spack/modules/linux-ubuntu14.04-x86_64/zlib/1.2.11-gcc-7.2.0-linux-ubuntu14.04-x86_64-co2px3k: - - module-whatis A free, general-purpose, legally unencumbered lossless data-compression library. - prepend-path MANPATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/share/man - prepend-path LIBRARY_PATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/lib - prepend-path LD_LIBRARY_PATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/lib - prepend-path CPATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/include - prepend-path PKG_CONFIG_PATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/lib/pkgconfig - prepend-path CMAKE_PREFIX_PATH /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/ - ------------------------------------------------------------------- - - $ echo $LD_LIBRARY_PATH - - $ module load zlib - $ echo $LD_LIBRARY_PATH - /home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu14.04-x86_64/gcc-7.2.0/zlib-1.2.11-co2px3k53m76lm6tofylh2mur2hnicux/lib - -and to undo the modifications when the same module file is unloaded: - - .. code-block:: console - - $ module unload zlib - $ echo $LD_LIBRARY_PATH - - $ - -Different formats exist for module files, and different tools -provide various levels of support for them. Spack can natively generate: - -1. Non-hierarchical module files written in TCL -2. Hierarchical module files written in Lua - -and can build `environment-modules <http://modules.sourceforge.net/>`_ -and `lmod <http://lmod.readthedocs.io/en/latest>`_ as support tools. -Which of the formats or tools best suits one's needs depends on each particular -use-case. For the sake of illustration, we'll be working on -both formats using ``lmod``. - -.. seealso:: - Environment modules - This is the original tool that provided modules support. Its first - version was coded in C in the early '90s and was later substituted by a version - completely coded in TCL - the one Spack is distributing. More details on - its features are given in the `homepage of the project <http://modules.sourceforge.net/>`_ - or in its `github page <https://github.com/cea-hpc/modules>`_. The tool is able to - interpret the non-hierarchical TCL modulefiles written by Spack. - - Lmod - Lmod is a module system written in Lua, designed to easily handle hierarchies of - module files. It's a drop-in replacement of Environment Modules and works with - both of the module file formats generated by Spack. - Despite being fully compatible with Environment Modules there are many features that - are unique to Lmod. These features are either - `targeted towards safety <http://lmod.readthedocs.io/en/latest/010_user.html#safety-features>`_ - or meant to - `extend the module system functionality <http://lmod.readthedocs.io/en/latest/010_user.html#module-hierarchy>`_. - - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -How do we generate module files? -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Before we dive into the hands-on sections it's worth spending a couple of words to explain how -module files are generated by Spack. The following diagram provides a high-level view -of the process: - - -.. image:: module_file_generation.* - -The red dashed line above represents Spack's boundaries, the blue one Spack's dependencies [#f1]_. -Module files are generated by combining: - - * the configuration details in ``config.yaml`` and ``modules.yaml`` - * the information contained in Spack packages (and processed by the module subpackage) - * a set of template files - -with `Jinja2 <http://jinja.pocoo.org/docs/2.9/>`_, an external template engine -that stamps out each particular module file. As Spack serves very diverse needs -this process has many points of customization, and we'll explore most of -them in the next sections. - -.. [#f1] Spack vendors its dependencies! This means that Spack comes with a copy of - each one of its dependencies, including ``Jinja2``, and is already configured to use them. - -.. _module_file_tutorial_prerequisites: - ----------------------- -Setup for the tutorial ----------------------- - -In order to showcase the capabilities of Spack's module file generation, we need -a representative set of software to work with. This set must include different -flavors of the same packages installed alongside each other and some -:ref:`external packages <sec-external-packages>`. - -The purpose of this setup is not to make our life harder but to demonstrate -how Spack can help with similar situations, as they will happen on real HPC clusters. -For instance, it's often preferable for Spack to use vendor-provided MPI -implementations than to build one itself. - -To keep the set of software we're dealing with manageable, we're going -to uninstall everything from earlier in the tutorial. - -.. code-block: console - - $ spack uninstall -ay - -^^^^^^^^^^^^^^^^^^^ -Build a module tool -^^^^^^^^^^^^^^^^^^^ - -The first thing that we need is the module tool. In this case we -choose ``lmod`` as it can work with both hierarchical and non-hierarchical -module file layouts. - -.. code-block:: console - - $ bin/spack install lmod - -Once the module tool is installed we need to have it available in the -current shell. As the installation directories are definitely not easy -to remember, we'll employ the command ``spack location`` to retrieve the -``lmod`` prefix directly from Spack: - -.. code-block:: console - - $ . $(spack location -i lmod)/lmod/lmod/init/bash - -Now we can re-source the setup file and Spack modules will be put in -our module path. - -.. code-block:: console - - $ . share/spack/setup-env.sh - -.. FIXME: this needs bootstrap support for ``lmod`` - -.. FIXME: check the docs here, update them if necessary - If you need to install Lmod or Environment module you can refer - to the documentation :ref:`here <InstallEnvironmentModules>`. - - -^^^^^^^^^^^^^^^^^^ -Add a new compiler -^^^^^^^^^^^^^^^^^^ - -The second step is to build a recent compiler. On first use, Spack -scans the environment and automatically locates the compiler(s) -already available on the system. For this tutorial, however, we want -to use ``gcc@7.2.0``. - - -.. code-block:: console - - $ spack install gcc@7.2.0 - ... - Wait a long time - ... - -Once ``gcc`` is installed we can use shell support to load it and make -it readily available: - -.. code-block:: console - - $ spack load gcc@7.2.0 - -It may not be apparent, but the last command employed the module files -generated automatically by Spack. What happens under the hood when you use -the ``spack load`` command is: - -1. the spec passed as argument is translated into a module file name -2. the current module tool is used to load that module file - -You can use this command to double check: - -.. code-block:: console - - $ module list - Currently Loaded Modules: - 1) gcc-7.2.0-gcc-5.4.0-b7smjjc - -Note that the 7-digit hash at the end of the generated module may vary depending -on architecture or package version. Now that we have ``gcc@7.2.0`` in ``PATH`` we -can finally add it to the list of compilers known to Spack: - -.. code-block:: console - - $ spack compiler add - ==> Added 1 new compiler to /home/spack1/.spack/linux/compilers.yaml - gcc@7.2.0 - ==> Compilers are defined in the following files: - /home/spack1/.spack/linux/compilers.yaml - - $ spack compiler list - ==> Available compilers - -- clang ubuntu16.04-x86_64 ------------------------------------- - clang@3.8.0-2ubuntu4 clang@3.7.1-2ubuntu2 - - -- gcc ubuntu16.04-x86_64 --------------------------------------- - gcc@7.2.0 gcc@5.4.0 gcc@4.7 - - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Build the software that will be used in the tutorial -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Finally, we should use Spack to install the packages used in the examples: - -.. code-block:: console - - $ spack install netlib-scalapack ^openmpi ^openblas - $ spack install netlib-scalapack ^mpich ^openblas - $ spack install netlib-scalapack ^openmpi ^netlib-lapack - $ spack install netlib-scalapack ^mpich ^netlib-lapack - $ spack install py-scipy ^openblas - - -.. _module_file_tutorial_non_hierarchical: - ------------------------------ -Non-hierarchical module files ------------------------------ - -If you arrived to this point you should have an environment that looks similar to: - -.. code-block:: console - - $ module avail - - ----------------------------------------------- /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64 ----------------------------------------------- - autoconf-2.69-gcc-5.4.0-3sx2gxe libsigsegv-2.11-gcc-7.2.0-g67xpfd openssl-1.0.2o-gcc-5.4.0-b4y3w3b - autoconf-2.69-gcc-7.2.0-yb2makb libtool-2.4.6-gcc-5.4.0-o2pfwjf openssl-1.0.2o-gcc-7.2.0-cvldq3v - automake-1.16.1-gcc-5.4.0-rymw7im libtool-2.4.6-gcc-7.2.0-kt2udm6 pcre-8.42-gcc-5.4.0-gt5lgzi - automake-1.16.1-gcc-7.2.0-qoowd5q libxml2-2.9.8-gcc-5.4.0-wpexsph perl-5.26.2-gcc-5.4.0-ic2kyoa - bzip2-1.0.6-gcc-5.4.0-ufczdvs libxml2-2.9.8-gcc-7.2.0-47gf5kk perl-5.26.2-gcc-7.2.0-fdwz5yu - bzip2-1.0.6-gcc-7.2.0-mwamumj lmod-7.8-gcc-5.4.0-kmhks3p pkgconf-1.4.2-gcc-5.4.0-fovrh7a - cmake-3.12.3-gcc-7.2.0-obqgn2v lua-5.3.4-gcc-5.4.0-cpfeo2w pkgconf-1.4.2-gcc-7.2.0-yoxwmgb - curl-7.60.0-gcc-5.4.0-vzqreb2 lua-luafilesystem-1_6_3-gcc-5.4.0-alakjim py-numpy-1.15.2-gcc-7.2.0-wbwtcxf - diffutils-3.6-gcc-5.4.0-2rhuivg lua-luaposix-33.4.0-gcc-5.4.0-7wqhwoc py-scipy-1.1.0-gcc-7.2.0-d5n3cph - diffutils-3.6-gcc-7.2.0-eauxwi7 m4-1.4.18-gcc-5.4.0-suf5jtc py-setuptools-40.4.3-gcc-7.2.0-5dbwfwn - expat-2.2.5-gcc-5.4.0-emyv67q m4-1.4.18-gcc-7.2.0-wdzvagl python-2.7.15-gcc-7.2.0-ucmr2mn - findutils-4.6.0-gcc-7.2.0-ca4b7zq mpc-1.1.0-gcc-5.4.0-iuf3gc3 readline-7.0-gcc-5.4.0-nxhwrg7 - gcc-7.2.0-gcc-5.4.0-b7smjjc (L) mpfr-3.1.6-gcc-5.4.0-jnt2nnp readline-7.0-gcc-7.2.0-ccruj2i - gdbm-1.14.1-gcc-5.4.0-q4fpyuo mpich-3.2.1-gcc-7.2.0-vt5xcat sqlite-3.23.1-gcc-7.2.0-5ltus3a - gdbm-1.14.1-gcc-7.2.0-zk5lhob ncurses-6.1-gcc-5.4.0-3o765ou tar-1.30-gcc-5.4.0-dk7lrpo - gettext-0.19.8.1-gcc-5.4.0-tawgous ncurses-6.1-gcc-7.2.0-xcgzqdv tcl-8.6.8-gcc-5.4.0-qhwyccy - git-2.19.1-gcc-5.4.0-p3gjnfa netlib-lapack-3.8.0-gcc-7.2.0-fj7nayd texinfo-6.5-gcc-7.2.0-cuqnfgf - gmp-6.1.2-gcc-5.4.0-qc4qcfz netlib-scalapack-2.0.2-gcc-7.2.0-67nmj7g unzip-6.0-gcc-5.4.0-ba23fbg - hwloc-1.11.9-gcc-7.2.0-gbyc65s netlib-scalapack-2.0.2-gcc-7.2.0-6jgjbyg util-macros-1.19.1-gcc-7.2.0-t62kozq - isl-0.18-gcc-5.4.0-vttqout netlib-scalapack-2.0.2-gcc-7.2.0-prgo67d xz-5.2.4-gcc-5.4.0-teneqii - libbsd-0.8.6-gcc-5.4.0-f4qkkwm netlib-scalapack-2.0.2-gcc-7.2.0-zxpt252 xz-5.2.4-gcc-7.2.0-rql5kog - libiconv-1.15-gcc-5.4.0-u2x3umv numactl-2.0.11-gcc-7.2.0-rifwktk zlib-1.2.11-gcc-5.4.0-5nus6kn - libpciaccess-0.13.5-gcc-7.2.0-riipwi2 openblas-0.3.3-gcc-7.2.0-xxoxfh4 zlib-1.2.11-gcc-7.2.0-ezuwp4p - libsigsegv-2.11-gcc-5.4.0-fypapcp openmpi-3.1.3-gcc-7.2.0-do5xfer - - Where: - L: Module is loaded - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - -The non-hierarchical module files that have been generated so far -follow :ref:`the default rules for module generation <modules-yaml>`. -Taking a look at the ``gcc`` module you'll see, for example: - -.. code-block:: console - - $ module show gcc-7.2.0-gcc-5.4.0-b7smjjc - ------------------------------------------------------------------------------------------------------------------------------------------------------------------- - /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64/gcc-7.2.0-gcc-5.4.0-b7smjjc: - ------------------------------------------------------------------------------------------------------------------------------------------------------------------- - whatis("The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages. ") - prepend_path("PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin") - prepend_path("MANPATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/share/man") - prepend_path("LD_LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/lib") - prepend_path("LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/lib") - prepend_path("LD_LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/lib64") - prepend_path("LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/lib64") - prepend_path("CPATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/include") - prepend_path("CMAKE_PREFIX_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/") - setenv("CC","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gcc") - setenv("CXX","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/g++") - setenv("FC","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - setenv("F77","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - setenv("F90","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C, - Fortran, Ada, and Go, as well as libraries for these languages. - ]]) - -As expected, a few environment variables representing paths will be modified -by the module file according to the default prefix inspection rules. - - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Filter unwanted modifications to the environment -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Now consider the case that your site has decided that ``CPATH`` and -``LIBRARY_PATH`` modifications should not be present in module files. What you can -do to abide by the rules is to create a configuration file ``~/.spack/modules.yaml`` -with the following content: - -.. code-block:: yaml - - modules: - tcl: - all: - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - -Next you should regenerate all the module files: - -.. code-block:: console - - $ spack module tcl refresh - ==> You are about to regenerate tcl module files for: - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - 3sx2gxe autoconf@2.69 b7smjjc gcc@7.2.0 f4qkkwm libbsd@0.8.6 cpfeo2w lua@5.3.4 3o765ou ncurses@6.1 dk7lrpo tar@1.30 - rymw7im automake@1.16.1 q4fpyuo gdbm@1.14.1 u2x3umv libiconv@1.15 alakjim lua-luafilesystem@1_6_3 b4y3w3b openssl@1.0.2o qhwyccy tcl@8.6.8 - ufczdvs bzip2@1.0.6 tawgous gettext@0.19.8.1 fypapcp libsigsegv@2.11 7wqhwoc lua-luaposix@33.4.0 gt5lgzi pcre@8.42 ba23fbg unzip@6.0 - vzqreb2 curl@7.60.0 p3gjnfa git@2.19.1 o2pfwjf libtool@2.4.6 suf5jtc m4@1.4.18 ic2kyoa perl@5.26.2 teneqii xz@5.2.4 - 2rhuivg diffutils@3.6 qc4qcfz gmp@6.1.2 wpexsph libxml2@2.9.8 iuf3gc3 mpc@1.1.0 fovrh7a pkgconf@1.4.2 5nus6kn zlib@1.2.11 - emyv67q expat@2.2.5 vttqout isl@0.18 kmhks3p lmod@7.8 jnt2nnp mpfr@3.1.6 nxhwrg7 readline@7.0 - - -- linux-ubuntu16.04-x86_64 / gcc@7.2.0 ------------------------- - yb2makb autoconf@2.69 riipwi2 libpciaccess@0.13.5 6jgjbyg netlib-scalapack@2.0.2 fdwz5yu perl@5.26.2 cuqnfgf texinfo@6.5 - qoowd5q automake@1.16.1 g67xpfd libsigsegv@2.11 zxpt252 netlib-scalapack@2.0.2 yoxwmgb pkgconf@1.4.2 t62kozq util-macros@1.19.1 - mwamumj bzip2@1.0.6 kt2udm6 libtool@2.4.6 67nmj7g netlib-scalapack@2.0.2 wbwtcxf py-numpy@1.15.2 rql5kog xz@5.2.4 - obqgn2v cmake@3.12.3 47gf5kk libxml2@2.9.8 prgo67d netlib-scalapack@2.0.2 d5n3cph py-scipy@1.1.0 ezuwp4p zlib@1.2.11 - eauxwi7 diffutils@3.6 wdzvagl m4@1.4.18 rifwktk numactl@2.0.11 5dbwfwn py-setuptools@40.4.3 - ca4b7zq findutils@4.6.0 vt5xcat mpich@3.2.1 xxoxfh4 openblas@0.3.3 ucmr2mn python@2.7.15 - zk5lhob gdbm@1.14.1 xcgzqdv ncurses@6.1 do5xfer openmpi@3.1.3 ccruj2i readline@7.0 - gbyc65s hwloc@1.11.9 fj7nayd netlib-lapack@3.8.0 cvldq3v openssl@1.0.2o 5ltus3a sqlite@3.23.1 - - ==> Do you want to proceed? [y/n] y - ==> Regenerating tcl module files - -If you take a look now at the module for ``gcc`` you'll see that the unwanted -paths have disappeared: - -.. code-block:: console - - $ module show gcc-7.2.0-gcc-5.4.0-b7smjjc - ------------------------------------------------------------------------------------------------------------------------------------------------------------------- - /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64/gcc-7.2.0-gcc-5.4.0-b7smjjc: - ------------------------------------------------------------------------------------------------------------------------------------------------------------------- - whatis("The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages. ") - prepend_path("PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin") - prepend_path("MANPATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/share/man") - prepend_path("LD_LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/lib") - prepend_path("LD_LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/lib64") - prepend_path("CMAKE_PREFIX_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/") - setenv("CC","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gcc") - setenv("CXX","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/g++") - setenv("FC","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - setenv("F77","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - setenv("F90","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C, - Fortran, Ada, and Go, as well as libraries for these languages. - ]]) - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Prevent some module files from being generated -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Another common request at many sites is to avoid exposing software that -is only needed as an intermediate step when building a newer stack. -Let's try to prevent the generation of -module files for anything that is compiled with ``gcc@5.4.0`` (the OS provided compiler). - -To do this you should add a ``blacklist`` keyword to ``~/.spack/modules.yaml``: - -.. code-block:: yaml - :emphasize-lines: 3,4 - - modules: - tcl: - blacklist: - - '%gcc@5.4.0' - all: - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - -and regenerate the module files: - -This time it is convenient to pass the option ``--delete-tree`` to the command that -regenerates the module files to instruct it to delete the existing tree and regenerate -a new one instead of overwriting the files in the existing directory. - -.. code-block:: console - - $ spack module tcl refresh --delete-tree - ==> You are about to regenerate tcl module files for: - - -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- - 3sx2gxe autoconf@2.69 b7smjjc gcc@7.2.0 f4qkkwm libbsd@0.8.6 cpfeo2w lua@5.3.4 3o765ou ncurses@6.1 dk7lrpo tar@1.30 - rymw7im automake@1.16.1 q4fpyuo gdbm@1.14.1 u2x3umv libiconv@1.15 alakjim lua-luafilesystem@1_6_3 b4y3w3b openssl@1.0.2o qhwyccy tcl@8.6.8 - ufczdvs bzip2@1.0.6 tawgous gettext@0.19.8.1 fypapcp libsigsegv@2.11 7wqhwoc lua-luaposix@33.4.0 gt5lgzi pcre@8.42 ba23fbg unzip@6.0 - vzqreb2 curl@7.60.0 p3gjnfa git@2.19.1 o2pfwjf libtool@2.4.6 suf5jtc m4@1.4.18 ic2kyoa perl@5.26.2 teneqii xz@5.2.4 - 2rhuivg diffutils@3.6 qc4qcfz gmp@6.1.2 wpexsph libxml2@2.9.8 iuf3gc3 mpc@1.1.0 fovrh7a pkgconf@1.4.2 5nus6kn zlib@1.2.11 - emyv67q expat@2.2.5 vttqout isl@0.18 kmhks3p lmod@7.8 jnt2nnp mpfr@3.1.6 nxhwrg7 readline@7.0 - - -- linux-ubuntu16.04-x86_64 / gcc@7.2.0 ------------------------- - yb2makb autoconf@2.69 riipwi2 libpciaccess@0.13.5 6jgjbyg netlib-scalapack@2.0.2 fdwz5yu perl@5.26.2 cuqnfgf texinfo@6.5 - qoowd5q automake@1.16.1 g67xpfd libsigsegv@2.11 zxpt252 netlib-scalapack@2.0.2 yoxwmgb pkgconf@1.4.2 t62kozq util-macros@1.19.1 - mwamumj bzip2@1.0.6 kt2udm6 libtool@2.4.6 67nmj7g netlib-scalapack@2.0.2 wbwtcxf py-numpy@1.15.2 rql5kog xz@5.2.4 - obqgn2v cmake@3.12.3 47gf5kk libxml2@2.9.8 prgo67d netlib-scalapack@2.0.2 d5n3cph py-scipy@1.1.0 ezuwp4p zlib@1.2.11 - eauxwi7 diffutils@3.6 wdzvagl m4@1.4.18 rifwktk numactl@2.0.11 5dbwfwn py-setuptools@40.4.3 - ca4b7zq findutils@4.6.0 vt5xcat mpich@3.2.1 xxoxfh4 openblas@0.3.3 ucmr2mn python@2.7.15 - zk5lhob gdbm@1.14.1 xcgzqdv ncurses@6.1 do5xfer openmpi@3.1.3 ccruj2i readline@7.0 - gbyc65s hwloc@1.11.9 fj7nayd netlib-lapack@3.8.0 cvldq3v openssl@1.0.2o 5ltus3a sqlite@3.23.1 - - ==> Do you want to proceed? [y/n] y - ==> Regenerating tcl module files - - $ module avail - - ----------------------------------------------- /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64 ----------------------------------------------- - autoconf-2.69-gcc-7.2.0-yb2makb m4-1.4.18-gcc-7.2.0-wdzvagl perl-5.26.2-gcc-7.2.0-fdwz5yu - automake-1.16.1-gcc-7.2.0-qoowd5q mpich-3.2.1-gcc-7.2.0-vt5xcat pkgconf-1.4.2-gcc-7.2.0-yoxwmgb - bzip2-1.0.6-gcc-7.2.0-mwamumj ncurses-6.1-gcc-7.2.0-xcgzqdv py-numpy-1.15.2-gcc-7.2.0-wbwtcxf - cmake-3.12.3-gcc-7.2.0-obqgn2v netlib-lapack-3.8.0-gcc-7.2.0-fj7nayd py-scipy-1.1.0-gcc-7.2.0-d5n3cph - diffutils-3.6-gcc-7.2.0-eauxwi7 netlib-scalapack-2.0.2-gcc-7.2.0-67nmj7g py-setuptools-40.4.3-gcc-7.2.0-5dbwfwn - findutils-4.6.0-gcc-7.2.0-ca4b7zq netlib-scalapack-2.0.2-gcc-7.2.0-6jgjbyg python-2.7.15-gcc-7.2.0-ucmr2mn - gdbm-1.14.1-gcc-7.2.0-zk5lhob netlib-scalapack-2.0.2-gcc-7.2.0-prgo67d readline-7.0-gcc-7.2.0-ccruj2i - hwloc-1.11.9-gcc-7.2.0-gbyc65s netlib-scalapack-2.0.2-gcc-7.2.0-zxpt252 sqlite-3.23.1-gcc-7.2.0-5ltus3a - libpciaccess-0.13.5-gcc-7.2.0-riipwi2 numactl-2.0.11-gcc-7.2.0-rifwktk texinfo-6.5-gcc-7.2.0-cuqnfgf - libsigsegv-2.11-gcc-7.2.0-g67xpfd openblas-0.3.3-gcc-7.2.0-xxoxfh4 util-macros-1.19.1-gcc-7.2.0-t62kozq - libtool-2.4.6-gcc-7.2.0-kt2udm6 openmpi-3.1.3-gcc-7.2.0-do5xfer xz-5.2.4-gcc-7.2.0-rql5kog - libxml2-2.9.8-gcc-7.2.0-47gf5kk openssl-1.0.2o-gcc-7.2.0-cvldq3v zlib-1.2.11-gcc-7.2.0-ezuwp4p - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - -If you look closely you'll see though that we went too far in blacklisting modules: -the module for ``gcc@7.2.0`` disappeared as it was bootstrapped with ``gcc@5.4.0``. To specify -exceptions to the blacklist rules you can use ``whitelist``: - -.. code-block:: yaml - :emphasize-lines: 3,4 - - modules: - tcl: - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - all: - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - -``whitelist`` rules always have precedence over ``blacklist`` rules. If you regenerate the modules again: - -.. code-block:: console - - $ spack module tcl refresh -y - ==> Regenerating tcl module files - - -you'll see that now the module for ``gcc@7.2.0`` has reappeared: - -.. code-block:: console - - $ module avail gcc-7.2.0-gcc-5.4.0-b7smjjc - - -------------------------------------------- /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64 --------------------------------------------- - gcc-7.2.0-gcc-5.4.0-b7smjjc - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - -An additional possibility that you can leverage to unclutter the environment -is that of preventing the generation of module files for implicitly installed -packages. In this case all one needs to do is to add the following line: - -.. code-block:: yaml - :emphasize-lines: 3 - - modules: - tcl: - blacklist_implicits: true - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - all: - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - -to ``modules.yaml`` and regenerate the module file tree as above. - -^^^^^^^^^^^^^^^^^^^^^^^^^ -Change module file naming -^^^^^^^^^^^^^^^^^^^^^^^^^ - -The next step in making module files more user-friendly is to -improve their naming scheme. -To reduce the length of the hash or remove it altogether you can -use the ``hash_length`` keyword in the configuration file: - -.. code-block:: yaml - :emphasize-lines: 3 - - modules: - tcl: - hash_length: 0 - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - all: - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - -If you try to regenerate the module files now you will get an error: - -.. code-block:: console - - $ spack module tcl refresh --delete-tree -y - ==> Error: Name clashes detected in module files: - - file: /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64/netlib-scalapack-2.0.2-gcc-7.2.0 - spec: netlib-scalapack@2.0.2%gcc@7.2.0 build_type=RelWithDebInfo ~pic+shared arch=linux-ubuntu16.04-x86_64 - spec: netlib-scalapack@2.0.2%gcc@7.2.0 build_type=RelWithDebInfo ~pic+shared arch=linux-ubuntu16.04-x86_64 - spec: netlib-scalapack@2.0.2%gcc@7.2.0 build_type=RelWithDebInfo ~pic+shared arch=linux-ubuntu16.04-x86_64 - spec: netlib-scalapack@2.0.2%gcc@7.2.0 build_type=RelWithDebInfo ~pic+shared arch=linux-ubuntu16.04-x86_64 - - ==> Error: Operation aborted - -.. note:: - We try to check for errors upfront! - In Spack we check for errors upfront whenever possible, so don't worry about your module files: - as a name clash was detected nothing has been changed on disk. - -The problem here is that without -the hashes the four different flavors of ``netlib-scalapack`` map to the same module file -name. We can add suffixes to differentiate them: - -.. code-block:: yaml - :emphasize-lines: 9-11,14-17 - - modules: - tcl: - hash_length: 0 - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - all: - suffixes: - '^openblas': openblas - '^netlib-lapack': netlib - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - netlib-scalapack: - suffixes: - '^openmpi': openmpi - '^mpich': mpich - -As you can see it is possible to specify rules that apply only to a -restricted set of packages using :ref:`anonymous specs <anonymous_specs>`. -Regenerating module files now we obtain: - -.. code-block:: console - - $ spack module tcl refresh --delete-tree -y - ==> Regenerating tcl module files - $ module avail - - ----------------------------------------------- /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64 ----------------------------------------------- - autoconf-2.69-gcc-7.2.0 m4-1.4.18-gcc-7.2.0 pkgconf-1.4.2-gcc-7.2.0 - automake-1.16.1-gcc-7.2.0 mpich-3.2.1-gcc-7.2.0 py-numpy-1.15.2-gcc-7.2.0-openblas - bzip2-1.0.6-gcc-7.2.0 ncurses-6.1-gcc-7.2.0 py-scipy-1.1.0-gcc-7.2.0-openblas - cmake-3.12.3-gcc-7.2.0 netlib-lapack-3.8.0-gcc-7.2.0 py-setuptools-40.4.3-gcc-7.2.0 - diffutils-3.6-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-netlib-mpich python-2.7.15-gcc-7.2.0 - findutils-4.6.0-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-netlib-openmpi readline-7.0-gcc-7.2.0 - gcc-7.2.0-gcc-5.4.0 netlib-scalapack-2.0.2-gcc-7.2.0-openblas-mpich sqlite-3.23.1-gcc-7.2.0 - gdbm-1.14.1-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-openblas-openmpi texinfo-6.5-gcc-7.2.0 - hwloc-1.11.9-gcc-7.2.0 numactl-2.0.11-gcc-7.2.0 util-macros-1.19.1-gcc-7.2.0 - libpciaccess-0.13.5-gcc-7.2.0 openblas-0.3.3-gcc-7.2.0 xz-5.2.4-gcc-7.2.0 - libsigsegv-2.11-gcc-7.2.0 openmpi-3.1.3-gcc-7.2.0 zlib-1.2.11-gcc-7.2.0 - libtool-2.4.6-gcc-7.2.0 openssl-1.0.2o-gcc-7.2.0 - libxml2-2.9.8-gcc-7.2.0 perl-5.26.2-gcc-7.2.0 - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - -Finally we can set a ``naming_scheme`` to prevent users from loading -modules that refer to different flavors of the same library/application: - -.. code-block:: yaml - :emphasize-lines: 4,10,11 - - modules: - tcl: - hash_length: 0 - naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}' - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - all: - conflict: - - '{name}' - suffixes: - '^openblas': openblas - '^netlib-lapack': netlib - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - netlib-scalapack: - suffixes: - '^openmpi': openmpi - '^mpich': mpich - -The final result should look like: - -.. code-block:: console - - $ spack module tcl refresh --delete-tree -y - ==> Regenerating tcl module files - $ module avail - - ----------------------------------------------- /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64 ----------------------------------------------- - autoconf/2.69-gcc-7.2.0 m4/1.4.18-gcc-7.2.0 pkgconf/1.4.2-gcc-7.2.0 - automake/1.16.1-gcc-7.2.0 mpich/3.2.1-gcc-7.2.0 py-numpy/1.15.2-gcc-7.2.0-openblas - bzip2/1.0.6-gcc-7.2.0 ncurses/6.1-gcc-7.2.0 py-scipy/1.1.0-gcc-7.2.0-openblas - cmake/3.12.3-gcc-7.2.0 netlib-lapack/3.8.0-gcc-7.2.0 py-setuptools/40.4.3-gcc-7.2.0 - diffutils/3.6-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-netlib-mpich python/2.7.15-gcc-7.2.0 - findutils/4.6.0-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-netlib-openmpi readline/7.0-gcc-7.2.0 - gcc/7.2.0-gcc-5.4.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-mpich sqlite/3.23.1-gcc-7.2.0 - gdbm/1.14.1-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-openmpi (D) texinfo/6.5-gcc-7.2.0 - hwloc/1.11.9-gcc-7.2.0 numactl/2.0.11-gcc-7.2.0 util-macros/1.19.1-gcc-7.2.0 - libpciaccess/0.13.5-gcc-7.2.0 openblas/0.3.3-gcc-7.2.0 xz/5.2.4-gcc-7.2.0 - libsigsegv/2.11-gcc-7.2.0 openmpi/3.1.3-gcc-7.2.0 zlib/1.2.11-gcc-7.2.0 - libtool/2.4.6-gcc-7.2.0 openssl/1.0.2o-gcc-7.2.0 - libxml2/2.9.8-gcc-7.2.0 perl/5.26.2-gcc-7.2.0 - - Where: - D: Default Module - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - -.. note:: - TCL specific directive - The directives ``naming_scheme`` and ``conflict`` are TCL specific and - can't be used in the ``lmod`` section of the configuration file. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Add custom environment modifications -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -At many sites it is customary to set an environment variable in a -package's module file that points to the folder in which the package -is installed. You can achieve this with Spack by adding an -``environment`` directive to the configuration file: - -.. code-block:: yaml - :emphasize-lines: 17-19 - - modules: - tcl: - hash_length: 0 - naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}' - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - all: - conflict: - - '{name}' - suffixes: - '^openblas': openblas - '^netlib-lapack': netlib - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - environment: - set: - '{name}_ROOT': '{prefix}' - netlib-scalapack: - suffixes: - '^openmpi': openmpi - '^mpich': mpich - -Under the hood Spack uses the :meth:`~spack.spec.Spec.format` API to substitute -tokens in either environment variable names or values. There are two caveats though: - -- The set of allowed tokens in variable names is restricted to - ``name``, ``version``, ``compiler``, ``compiler.name``, - ``compiler.version``, ``architecture`` -- Any token expanded in a variable name is made uppercase, but other than that - case sensitivity is preserved - -Regenerating the module files results in something like: - -.. code-block:: console - :emphasize-lines: 15 - - $ spack module tcl refresh -y - ==> Regenerating tcl module files - - $ module show gcc - ------------------------------------------------------------------------------------------------------------------------------------------------------------------- - /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64/gcc/7.2.0-gcc-5.4.0: - ------------------------------------------------------------------------------------------------------------------------------------------------------------------- - whatis("The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages. ") - conflict("gcc") - prepend_path("PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin") - prepend_path("MANPATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/share/man") - prepend_path("LD_LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/lib") - prepend_path("LD_LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/lib64") - prepend_path("CMAKE_PREFIX_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/") - setenv("CC","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gcc") - setenv("CXX","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/g++") - setenv("FC","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - setenv("F77","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - setenv("F90","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - setenv("GCC_ROOT","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs") - help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C, - Fortran, Ada, and Go, as well as libraries for these languages. - ]]) - -As you can see, the ``gcc`` module has the environment variable ``GCC_ROOT`` set. - -Sometimes it's also useful to apply environment modifications selectively and target -only certain packages. You can, for instance set the common variables ``CC``, ``CXX``, -etc. in the ``gcc`` module file and apply other custom modifications to the -``openmpi`` modules as follows: - -.. code-block:: yaml - :emphasize-lines: 20-32 - - modules: - tcl: - hash_length: 0 - naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}' - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - all: - conflict: - - '{name}' - suffixes: - '^openblas': openblas - '^netlib-lapack': netlib - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - environment: - set: - '{name}_ROOT': '{prefix}' - gcc: - environment: - set: - CC: gcc - CXX: g++ - FC: gfortran - F90: gfortran - F77: gfortran - openmpi: - environment: - set: - SLURM_MPI_TYPE: pmi2 - OMPI_MCA_btl_openib_warn_default_gid_prefix: '0' - netlib-scalapack: - suffixes: - '^openmpi': openmpi - '^mpich': mpich - -This time we will be more selective and regenerate only the ``gcc`` and -``openmpi`` module files: - -.. code-block:: console - - $ spack module tcl refresh -y gcc - ==> Regenerating tcl module files - - $ spack module tcl refresh -y openmpi - ==> Regenerating tcl module files - - $ module show gcc - ------------------------------------------------------------------------------------------------------------------------------------------------------------------- - /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64/gcc/7.2.0-gcc-5.4.0: - ------------------------------------------------------------------------------------------------------------------------------------------------------------------- - whatis("The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages. ") - conflict("gcc") - prepend_path("PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin") - prepend_path("MANPATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/share/man") - prepend_path("LD_LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/lib") - prepend_path("LD_LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/lib64") - prepend_path("CMAKE_PREFIX_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/") - setenv("CC","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gcc") - setenv("CXX","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/g++") - setenv("FC","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - setenv("F77","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - setenv("F90","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs/bin/gfortran") - setenv("GCC_ROOT","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gcc-7.2.0-b7smjjcsmwe5u5fcsvjmonlhlzzctnfs") - setenv("CC","gcc") - setenv("CXX","g++'") - setenv("FC","gfortran") - setenv("F77","gfortran") - setenv("F90","gfortran") - help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C, - Fortran, Ada, and Go, as well as libraries for these languages. - ]]) - - $ module show openmpi - ------------------------------------------------------------------------------------------------------------------------------------------------------------------- - /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64/openmpi/3.1.3-gcc-7.2.0: - ------------------------------------------------------------------------------------------------------------------------------------------------------------------- - whatis("An open source Message Passing Interface implementation. ") - conflict("openmpi") - prepend_path("PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/openmpi-3.1.3-do5xfer2whhk7gc26atgs3ozr3ljbvs4/bin") - prepend_path("MANPATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/openmpi-3.1.3-do5xfer2whhk7gc26atgs3ozr3ljbvs4/share/man") - prepend_path("LD_LIBRARY_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/openmpi-3.1.3-do5xfer2whhk7gc26atgs3ozr3ljbvs4/lib") - prepend_path("PKG_CONFIG_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/openmpi-3.1.3-do5xfer2whhk7gc26atgs3ozr3ljbvs4/lib/pkgconfig") - prepend_path("CMAKE_PREFIX_PATH","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/openmpi-3.1.3-do5xfer2whhk7gc26atgs3ozr3ljbvs4/") - setenv("OPENMPI_ROOT","/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/openmpi-3.1.3-do5xfer2whhk7gc26atgs3ozr3ljbvs4") - setenv("SLURM_MPI_TYPE","pmi2") - setenv("OMPI_MCA_btl_openib_warn_default_gid_prefix","0") - help([[An open source Message Passing Interface implementation. The Open MPI - Project is an open source Message Passing Interface implementation that - is developed and maintained by a consortium of academic, research, and - industry partners. Open MPI is therefore able to combine the expertise, - technologies, and resources from all across the High Performance - Computing community in order to build the best MPI library available. - Open MPI offers advantages for system and software vendors, application - developers and computer science researchers. - ]]) - - -^^^^^^^^^^^^^^^^^^^^^ -Autoload dependencies -^^^^^^^^^^^^^^^^^^^^^ - -Spack can also generate module files that contain code to load the -dependencies automatically. You can, for instance generate python -modules that load their dependencies by adding the ``autoload`` -directive and assigning it the value ``direct``: - -.. code-block:: yaml - :emphasize-lines: 3,38,39 - - modules: - tcl: - verbose: True - hash_length: 0 - naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}' - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - all: - conflict: - - '{name}' - suffixes: - '^openblas': openblas - '^netlib-lapack': netlib - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - environment: - set: - '{name}_ROOT': '{prefix}' - gcc: - environment: - set: - CC: gcc - CXX: g++ - FC: gfortran - F90: gfortran - F77: gfortran - openmpi: - environment: - set: - SLURM_MPI_TYPE: pmi2 - OMPI_MCA_btl_openib_warn_default_gid_prefix: '0' - netlib-scalapack: - suffixes: - '^openmpi': openmpi - '^mpich': mpich - ^python: - autoload: 'direct' - -and regenerating the module files for every package that depends on ``python``: - -.. code-block:: console - - root@module-file-tutorial:/# spack module tcl refresh -y ^python - ==> Regenerating tcl module files - -Now the ``py-scipy`` module will be: - -.. code-block:: tcl - - #%Module1.0 - ## Module file created by spack (https://github.com/spack/spack) on 2018-11-11 22:10:48.834221 - ## - ## py-scipy@1.1.0%gcc@7.2.0 arch=linux-ubuntu16.04-x86_64 /d5n3cph - ## - - - module-whatis "SciPy (pronounced 'Sigh Pie') is a Scientific Library for Python. It provides many user-friendly and efficient numerical routines such as routines for numerical integration and optimization." - - proc ModulesHelp { } { - puts stderr "SciPy (pronounced "Sigh Pie") is a Scientific Library for Python. It" - puts stderr "provides many user-friendly and efficient numerical routines such as" - puts stderr "routines for numerical integration and optimization." - } - - if { [ module-info mode load ] && ![ is-loaded python/2.7.15-gcc-7.2.0 ] } { - puts stderr "Autoloading python/2.7.15-gcc-7.2.0" - module load python/2.7.15-gcc-7.2.0 - } - if { [ module-info mode load ] && ![ is-loaded openblas/0.3.3-gcc-7.2.0 ] } { - puts stderr "Autoloading openblas/0.3.3-gcc-7.2.0" - module load openblas/0.3.3-gcc-7.2.0 - } - if { [ module-info mode load ] && ![ is-loaded py-numpy/1.15.2-gcc-7.2.0-openblas ] } { - puts stderr "Autoloading py-numpy/1.15.2-gcc-7.2.0-openblas" - module load py-numpy/1.15.2-gcc-7.2.0-openblas - } - conflict py-scipy - - prepend-path LD_LIBRARY_PATH "/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/py-scipy-1.1.0-d5n3cphk2lx2v74ypwb6h7tna7vvgdyn/lib" - prepend-path CMAKE_PREFIX_PATH "/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/py-scipy-1.1.0-d5n3cphk2lx2v74ypwb6h7tna7vvgdyn/" - prepend-path PYTHONPATH "/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/py-scipy-1.1.0-d5n3cphk2lx2v74ypwb6h7tna7vvgdyn/lib/python2.7/site-packages" - setenv PY_SCIPY_ROOT "/home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/py-scipy-1.1.0-d5n3cphk2lx2v74ypwb6h7tna7vvgdyn" - -and will contain code to autoload all the dependencies: - -.. code-block:: console - - $ module load py-scipy - Autoloading python/2.7.15-gcc-7.2.0 - Autoloading openblas/0.3.3-gcc-7.2.0 - Autoloading py-numpy/1.15.2-gcc-7.2.0-openblas - -In case messages are unwanted during the autoload procedure, it will be -sufficient to omit the line setting ``verbose: True`` in the configuration file above. - -------------------------- -Hierarchical module files -------------------------- - -So far we worked with non-hierarchical module files, i.e. with module files -that are all generated in the same root directory and don't attempt to -dynamically modify the ``MODULEPATH``. This results in a flat module structure where -all the software is visible at the same time: - -.. code-block:: console - - $ module avail - - ----------------------------------------------- /home/spack1/spack/share/spack/modules/linux-ubuntu16.04-x86_64 ----------------------------------------------- - autoconf/2.69-gcc-7.2.0 m4/1.4.18-gcc-7.2.0 pkgconf/1.4.2-gcc-7.2.0 - automake/1.16.1-gcc-7.2.0 mpich/3.2.1-gcc-7.2.0 py-numpy/1.15.2-gcc-7.2.0-openblas (L) - bzip2/1.0.6-gcc-7.2.0 ncurses/6.1-gcc-7.2.0 py-scipy/1.1.0-gcc-7.2.0-openblas (L) - cmake/3.12.3-gcc-7.2.0 netlib-lapack/3.8.0-gcc-7.2.0 py-setuptools/40.4.3-gcc-7.2.0 - diffutils/3.6-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-netlib-mpich python/2.7.15-gcc-7.2.0 (L) - findutils/4.6.0-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-netlib-openmpi readline/7.0-gcc-7.2.0 - gcc/7.2.0-gcc-5.4.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-mpich sqlite/3.23.1-gcc-7.2.0 - gdbm/1.14.1-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-openmpi (D) texinfo/6.5-gcc-7.2.0 - hwloc/1.11.9-gcc-7.2.0 numactl/2.0.11-gcc-7.2.0 util-macros/1.19.1-gcc-7.2.0 - libpciaccess/0.13.5-gcc-7.2.0 openblas/0.3.3-gcc-7.2.0 (L) xz/5.2.4-gcc-7.2.0 - libsigsegv/2.11-gcc-7.2.0 openmpi/3.1.3-gcc-7.2.0 zlib/1.2.11-gcc-7.2.0 - libtool/2.4.6-gcc-7.2.0 openssl/1.0.2o-gcc-7.2.0 - libxml2/2.9.8-gcc-7.2.0 perl/5.26.2-gcc-7.2.0 - - Where: - L: Module is loaded - D: Default Module - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - -This layout is quite simple to deploy, but you can see from the above snippet -that nothing prevents users from loading incompatible sets of modules: - -.. code-block:: console - - $ module purge - $ module load netlib-lapack/3.8.0-gcc-7.2.0 openblas/0.3.3-gcc-7.2.0 - $ module list - - Currently Loaded Modules: - 1) netlib-lapack/3.8.0-gcc-7.2.0 2) openblas/0.3.3-gcc-7.2.0 - -Even if ``conflicts`` directives are carefully placed in module files, they: - - - won't enforce a consistent environment, but will just report an error - - need constant updates, for instance as soon as a new compiler or MPI library is installed - -`Hierarchical module files <http://lmod.readthedocs.io/en/latest/080_hierarchy.html>`_ try to -overcome these shortcomings by showing at start-up only a restricted view of what is -available on the system: more specifically only the software that has been installed with -OS provided compilers. Among this software there will be other - usually more recent - compilers -that, once loaded, will prepend new directories to ``MODULEPATH`` unlocking all the software -that was compiled with them. This "unlocking" idea can then be extended arbitrarily to -virtual dependencies, as we'll see in the following section. - -^^^^^^^^^^^^^^^^^ -Core/Compiler/MPI -^^^^^^^^^^^^^^^^^ - -The most widely used hierarchy is the so called ``Core/Compiler/MPI`` where, on top -of the compilers, different MPI libraries also unlock software linked to them. -There are just a few steps needed to adapt the ``modules.yaml`` file we used previously: - - #. enable the ``lmod`` file generator - #. change the ``tcl`` tag to ``lmod`` - #. remove ``tcl`` specific directives (``naming_scheme`` and ``conflict``) - #. declare which compilers are considered ``core_compilers`` - #. remove the ``mpi`` related suffixes (as they will be substituted by hierarchies) - -After these modifications your configuration file should look like: - -.. code-block:: yaml - :emphasize-lines: 2-8 - - modules: - enable:: - - lmod - lmod: - core_compilers: - - 'gcc@5.4.0' - hierarchy: - - mpi - hash_length: 0 - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - all: - suffixes: - '^openblas': openblas - '^netlib-lapack': netlib - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - environment: - set: - '{name}_ROOT': '{prefix}' - gcc: - environment: - set: - CC: gcc - CXX: g++ - FC: gfortran - F90: gfortran - F77: gfortran - openmpi: - environment: - set: - SLURM_MPI_TYPE: pmi2 - OMPI_MCA_btl_openib_warn_default_gid_prefix: '0' - - -.. note:: - Double colon in configuration files - The double colon after ``enable`` is intentional and it serves the - purpose of overriding the default list of enabled generators so - that only ``lmod`` will be active (see :ref:`config-overrides` for more - details). - -The directive ``core_compilers`` accepts a list of compilers. Everything built -using these compilers will create a module in the ``Core`` part of the hierarchy, -which is the entry point for hierarchical module files. It is -common practice to put the OS provided compilers in the list and only build common utilities -and other compilers with them. - -If we now regenerate the module files: - -.. code-block:: console - - $ spack module lmod refresh --delete-tree -y - ==> Regenerating lmod module files - -and update ``MODULEPATH`` to point to the ``Core``: - -.. code-block:: console - - $ module purge - $ module unuse $HOME/spack/share/spack/modules/linux-ubuntu16.04-x86_64 - $ module use $HOME/spack/share/spack/lmod/linux-ubuntu16.04-x86_64/Core - -asking for the available modules will return: - -.. code-block:: console - - $ module avail - - ----------------------------------------------------------- share/spack/lmod/linux-ubuntu16.04-x86_64/Core ------------------------------------------------------------ - gcc/7.2.0 - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - -Unsurprisingly, the only visible module is ``gcc``. Loading that we'll unlock -the ``Compiler`` part of the hierarchy: - -.. code-block:: console - - $ module load gcc - $ module avail - - ------------------------------------------- /home/spack1/spack/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 -------------------------------------------- - autoconf/2.69 findutils/4.6.0 libtool/2.4.6 netlib-lapack/3.8.0 perl/5.26.2 python/2.7.15 xz/5.2.4 - automake/1.16.1 gdbm/1.14.1 libxml2/2.9.8 numactl/2.0.11 pkgconf/1.4.2 readline/7.0 zlib/1.2.11 - bzip2/1.0.6 hwloc/1.11.9 m4/1.4.18 openblas/0.3.3 py-numpy/1.15.2-openblas sqlite/3.23.1 - cmake/3.12.3 libpciaccess/0.13.5 mpich/3.2.1 openmpi/3.1.3 py-scipy/1.1.0-openblas texinfo/6.5 - diffutils/3.6 libsigsegv/2.11 ncurses/6.1 openssl/1.0.2o py-setuptools/40.4.3 util-macros/1.19.1 - - ----------------------------------------------------------- share/spack/lmod/linux-ubuntu16.04-x86_64/Core ------------------------------------------------------------ - gcc/7.2.0 (L) - - Where: - L: Module is loaded - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - -The same holds true also for the ``MPI`` part, that you can enable by loading -either ``mpich`` or ``openmpi``. Let's start by loading ``mpich``: - -.. code-block:: console - - $ module load mpich - $ module avail - - --------------------------------- /home/spack1/spack/share/spack/lmod/linux-ubuntu16.04-x86_64/mpich/3.2.1-vt5xcat/gcc/7.2.0 ---------------------------------- - netlib-scalapack/2.0.2-netlib netlib-scalapack/2.0.2-openblas (D) - - ------------------------------------------- /home/spack1/spack/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 -------------------------------------------- - autoconf/2.69 findutils/4.6.0 libtool/2.4.6 netlib-lapack/3.8.0 perl/5.26.2 python/2.7.15 xz/5.2.4 - automake/1.16.1 gdbm/1.14.1 libxml2/2.9.8 numactl/2.0.11 pkgconf/1.4.2 readline/7.0 zlib/1.2.11 - bzip2/1.0.6 hwloc/1.11.9 m4/1.4.18 openblas/0.3.3 py-numpy/1.15.2-openblas sqlite/3.23.1 - cmake/3.12.3 libpciaccess/0.13.5 mpich/3.2.1 (L) openmpi/3.1.3 py-scipy/1.1.0-openblas texinfo/6.5 - diffutils/3.6 libsigsegv/2.11 ncurses/6.1 openssl/1.0.2o py-setuptools/40.4.3 util-macros/1.19.1 - - ----------------------------------------------------------- share/spack/lmod/linux-ubuntu16.04-x86_64/Core ------------------------------------------------------------ - gcc/7.2.0 (L) - - Where: - L: Module is loaded - D: Default Module - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - - - root@module-file-tutorial:/# module load openblas netlib-scalapack/2.0.2-openblas - root@module-file-tutorial:/# module list - - Currently Loaded Modules: - 1) gcc/7.2.0 2) mpich/3.2.1 3) openblas/0.3.3 4) netlib-scalapack/2.0.2-openblas - -At this point we can showcase the improved consistency that a hierarchical layout provides -over a non-hierarchical one: - -.. code-block:: console - - $ module load openmpi - - Lmod is automatically replacing "mpich/3.2.1" with "openmpi/3.1.3". - - - Due to MODULEPATH changes, the following have been reloaded: - 1) netlib-scalapack/2.0.2-openblas - -``Lmod`` took care of swapping the MPI provider for us, and it also substituted the -``netlib-scalapack`` module to conform to the change in the MPI. -In this way we can't accidentally pull-in two different MPI providers at the -same time or load a module file for a package linked to ``openmpi`` when ``mpich`` is also loaded. -Consistency for compilers and MPI is ensured by the tool. - - -^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Add LAPACK to the hierarchy -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The hierarchy just shown is already a great improvement over non-hierarchical layouts, -but it still has an asymmetry: ``LAPACK`` providers cover the same semantic role -as ``MPI`` providers, but yet they are not part of the hierarchy. - -To be more practical, this means that although we have gained an improved consistency in -our environment when it comes to ``MPI``, we still have the same problems as we had before -for ``LAPACK`` implementations: - -.. code-block:: console - - root@module-file-tutorial:/# module list - - Currently Loaded Modules: - 1) gcc/7.2.0 2) openblas/0.3.3 3) openmpi/3.1.3 4) netlib-scalapack/2.0.2-openblas - - root@module-file-tutorial:/# module load netlib-scalapack/2.0.2-netlib - - The following have been reloaded with a version change: - 1) netlib-scalapack/2.0.2-openblas => netlib-scalapack/2.0.2-netlib - - root@module-file-tutorial:/# module list - - Currently Loaded Modules: - 1) gcc/7.2.0 2) openblas/0.3.3 3) openmpi/3.1.3 4) netlib-scalapack/2.0.2-netlib - -Hierarchies that are deeper than ``Core``/``Compiler``/``MPI`` are -probably still considered "unusual" or "impractical" at many sites, mainly because -module files are written manually and keeping track of the combinations -among multiple providers quickly becomes quite involved. - -For instance, having both ``MPI`` and ``LAPACK`` in the hierarchy -means we must classify software into one of four categories: - - #. Software that doesn't depend on ``MPI`` or ``LAPACK`` - #. Software that depends only on ``MPI`` - #. Software that depends only on ``LAPACK`` - #. Software that depends on both - -to decide when to show it to the user. The situation becomes more involved as the number of virtual -dependencies in the hierarchy increases. - -We can take advantage of the DAG that Spack maintains for the installed software and solve -this combinatorial problem in a clean and automated way. -In some sense Spack's ability to manage this combinatorial complexity makes deeper -hierarchies feasible. - -Coming back to our example, let's add ``lapack`` to the hierarchy and remove any remaining suffix: - -.. code-block:: yaml - :emphasize-lines: 9 - - modules: - enable:: - - lmod - lmod: - core_compilers: - - 'gcc@5.4.0' - hierarchy: - - mpi - - lapack - hash_length: 0 - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - all: - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - environment: - set: - '{name}_ROOT': '{prefix}' - gcc: - environment: - set: - CC: gcc - CXX: g++ - FC: gfortran - F90: gfortran - F77: gfortran - openmpi: - environment: - set: - SLURM_MPI_TYPE: pmi2 - OMPI_MCA_btl_openib_warn_default_gid_prefix: '0' - -After module files have been regenerated as usual: - -.. code-block:: console - - root@module-file-tutorial:/# module purge - - root@module-file-tutorial:/# spack module lmod refresh --delete-tree -y - ==> Regenerating lmod module files - -we can see that now we have additional components in the hierarchy: - -.. code-block:: console - - $ module load gcc - $ module load openblas - $ module avail - - -------------------------------- /home/spack1/spack/share/spack/lmod/linux-ubuntu16.04-x86_64/openblas/0.3.3-xxoxfh4/gcc/7.2.0 -------------------------------- - py-numpy/1.15.2 py-scipy/1.1.0 - - ------------------------------------------- /home/spack1/spack/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 -------------------------------------------- - autoconf/2.69 findutils/4.6.0 libtool/2.4.6 netlib-lapack/3.8.0 perl/5.26.2 sqlite/3.23.1 - automake/1.16.1 gdbm/1.14.1 libxml2/2.9.8 numactl/2.0.11 pkgconf/1.4.2 texinfo/6.5 - bzip2/1.0.6 hwloc/1.11.9 m4/1.4.18 openblas/0.3.3 (L) py-setuptools/40.4.3 util-macros/1.19.1 - cmake/3.12.3 libpciaccess/0.13.5 mpich/3.2.1 openmpi/3.1.3 python/2.7.15 xz/5.2.4 - diffutils/3.6 libsigsegv/2.11 ncurses/6.1 openssl/1.0.2o readline/7.0 zlib/1.2.11 - - ----------------------------------------------------------- share/spack/lmod/linux-ubuntu16.04-x86_64/Core ------------------------------------------------------------ - gcc/7.2.0 (L) - - Where: - L: Module is loaded - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - - - $ module load openmpi - $ module avail - - --------------------- /home/spack1/spack/share/spack/lmod/linux-ubuntu16.04-x86_64/openmpi/3.1.3-do5xfer/openblas/0.3.3-xxoxfh4/gcc/7.2.0 --------------------- - netlib-scalapack/2.0.2 - - -------------------------------- /home/spack1/spack/share/spack/lmod/linux-ubuntu16.04-x86_64/openblas/0.3.3-xxoxfh4/gcc/7.2.0 -------------------------------- - py-numpy/1.15.2 py-scipy/1.1.0 - - ------------------------------------------- /home/spack1/spack/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 -------------------------------------------- - autoconf/2.69 findutils/4.6.0 libtool/2.4.6 netlib-lapack/3.8.0 perl/5.26.2 sqlite/3.23.1 - automake/1.16.1 gdbm/1.14.1 libxml2/2.9.8 numactl/2.0.11 pkgconf/1.4.2 texinfo/6.5 - bzip2/1.0.6 hwloc/1.11.9 m4/1.4.18 openblas/0.3.3 (L) py-setuptools/40.4.3 util-macros/1.19.1 - cmake/3.12.3 libpciaccess/0.13.5 mpich/3.2.1 openmpi/3.1.3 (L) python/2.7.15 xz/5.2.4 - diffutils/3.6 libsigsegv/2.11 ncurses/6.1 openssl/1.0.2o readline/7.0 zlib/1.2.11 - - ---------------------------------------------- /home/spack1/spack/share/spack/lmod/linux-ubuntu16.04-x86_64/Core ---------------------------------------------- - gcc/7.2.0 (L) - - Where: - L: Module is loaded - - Use "module spider" to find all possible modules. - Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". - -Both ``MPI`` and ``LAPACK`` providers will now benefit from the same safety features: - -.. code-block:: console - - $ module load py-numpy netlib-scalapack - $ module load mpich - - Lmod is automatically replacing "openmpi/3.1.3" with "mpich/3.2.1". - - - Due to MODULEPATH changes, the following have been reloaded: - 1) netlib-scalapack/2.0.2 - - $ module load netlib-lapack - - Lmod is automatically replacing "openblas/0.3.3" with "netlib-lapack/3.8.0". - - - Inactive Modules: - 1) py-numpy - - Due to MODULEPATH changes, the following have been reloaded: - 1) netlib-scalapack/2.0.2 - -Because we only compiled ``py-numpy`` with ``openblas`` the module -is made inactive when we switch the ``LAPACK`` provider. The user -environment is now consistent by design! - ----------------------- -Working with templates ----------------------- - -As briefly mentioned in the introduction, Spack uses `Jinja2 <http://jinja.pocoo.org/docs/2.9/>`_ -to generate each individual module file. -This means that you have all of its flexibility and power when it comes to -customizing what gets generated! - -^^^^^^^^^^^^^^^^^^^^^ -Module file templates -^^^^^^^^^^^^^^^^^^^^^ - -The templates that Spack uses to generate module files are stored in the -``share/spack/templates/module`` directory within the Spack prefix, and -they all share the same common structure. Usually, they start with a -header that identifies the type of module being generated. In the case of -hierarchical module files it's: - -.. literalinclude:: _spack_root/share/spack/templates/modules/modulefile.lua - :language: jinja - :lines: 1-6 - -The statements within double curly brackets ``{{ ... }}`` denote -`expressions <http://jinja.pocoo.org/docs/2.9/templates/#expressions>`_ -that will be evaluated and substituted at module generation time. -The rest of the file is then divided into -`blocks <http://jinja.pocoo.org/docs/2.9/templates/#template-inheritance>`_ -that can be overridden or extended by users, if need be. -`Control structures <http://jinja.pocoo.org/docs/2.9/templates/#list-of-control-structures>`_ -, delimited by ``{% ... %}``, -are also permitted in the template language: - -.. literalinclude:: _spack_root/share/spack/templates/modules/modulefile.lua - :language: jinja - :lines: 73-88 - -The locations where Spack looks for templates are specified -in ``config.yaml``: - -.. literalinclude:: _spack_root/etc/spack/defaults/config.yaml - :language: yaml - :lines: 21-24 - -and can be extended by users to employ custom templates, as we'll see next. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Extend the default templates -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Let's assume one of our software is protected by group membership: -allowed users belong to the same linux group, and access is granted at group level. -Wouldn't it be nice if people that are not -yet entitled to use it could receive a helpful message at module load time -that tells them who to contact in your organization to be inserted in the group? - -To automate the generation of module files with such site-specific behavior -we'll start by extending the list of locations where Spack looks for module -files. Let's create the file ``~/.spack/config.yaml`` with the content: - -.. code-block:: yaml - - config: - template_dirs: - - $HOME/.spack/templates - -This tells Spack to also search another location when looking for template files. -Next, we need to create our custom template extension in the folder listed above: - -.. code-block:: jinja - - {% extends "modules/modulefile.lua" %} - {% block footer %} - -- Access is granted only to specific groups - if not isDir("{{ spec.prefix }}") then - LmodError ( - "You don't have the necessary rights to run \"{{ spec.name }}\".\n\n", - "\tPlease write an e-mail to 1234@foo.com if you need further information on how to get access to it.\n" - ) - end - {% endblock %} - -Let's name this file ``group-restricted.lua``. The line: - -.. code-block:: jinja - - {% extends "modules/modulefile.lua" %} - -tells Jinja2 that we are reusing the standard template for hierarchical module files. -The section: - -.. code-block:: jinja - - {% block footer %} - -- Access is granted only to specific groups - if not isDir("{{ spec.prefix }}") then - LmodError ( - "You don't have the necessary rights to run \"{{ spec.name }}\".\n\n", - "\tPlease write an e-mail to 1234@foo.com if you need further information on how to get access to it.\n" - ) - end - {% endblock %} - -overrides the ``footer`` block. -Finally, we need to add a couple of lines in ``modules.yaml`` to tell Spack which specs -need to use the new custom template. For the sake of illustration let's assume -it's ``netlib-scalapack``: - -.. code-block:: yaml - :emphasize-lines: 35-36 - - modules: - enable:: - - lmod - lmod: - core_compilers: - - 'gcc@5.4.0' - hierarchy: - - mpi - - lapack - hash_length: 0 - whitelist: - - gcc - blacklist: - - '%gcc@5.4.0' - - readline - all: - filter: - environment_blacklist: ['CPATH', 'LIBRARY_PATH'] - environment: - set: - '{name}_ROOT': '{prefix}' - gcc: - environment: - set: - CC: gcc - CXX: g++ - FC: gfortran - F90: gfortran - F77: gfortran - openmpi: - environment: - set: - SLURM_MPI_TYPE: pmi2 - OMPI_MCA_btl_openib_warn_default_gid_prefix: '0' - netlib-scalapack: - template: 'group-restricted.lua' - -If we regenerate the module files one last time: - -.. code-block:: console - - root@module-file-tutorial:/# spack module lmod refresh -y netlib-scalapack - ==> Regenerating lmod module files - -we'll find the following at the end of each ``netlib-scalapack`` module file: - -.. code-block:: lua - - -- Access is granted only to specific groups - if not isDir("/usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/netlib-scalapack-2.0.2-d3lertflood3twaor44eam2kcr4l72ag") then - LmodError ( - "You don't have the necessary rights to run \"netlib-scalapack\".\n\n", - "\tPlease write an e-mail to 1234@foo.com if you need further information on how to get access to it.\n" - ) - end - -and every user that doesn't have access to the software will now be redirected to -the right e-mail address where to ask for it! diff --git a/lib/spack/docs/tutorial_packaging.rst b/lib/spack/docs/tutorial_packaging.rst deleted file mode 100644 index ca693ba14f..0000000000 --- a/lib/spack/docs/tutorial_packaging.rst +++ /dev/null @@ -1,559 +0,0 @@ -.. Copyright 2013-2019 Lawrence Livermore National Security, LLC and other - Spack Project Developers. See the top-level COPYRIGHT file for details. - - SPDX-License-Identifier: (Apache-2.0 OR MIT) - -.. _packaging-tutorial: - -========================= -Package Creation Tutorial -========================= - -This tutorial will walk you through the steps behind building a simple -package installation script. We'll focus on writing a package for -mpileaks, an MPI debugging tool. By creating a package file we're -essentially giving Spack a recipe for how to build a particular piece of -software. We're describing some of the software's dependencies, where to -find the package, what commands and options are used to build the package -from source, and more. Once we've specified a package's recipe, we can -ask Spack to build that package in many different ways. - -This tutorial assumes you have a basic familiarity with some of the Spack -commands, and that you have a working version of Spack installed. If -not, we suggest looking at Spack's :ref:`getting_started` guide. This -tutorial also assumes you have at least a beginner's-level familiarity -with Python. - -Also note that this document is a tutorial. It can help you get started -with packaging, but is not intended to be complete. See Spack's -:ref:`packaging-guide` for more complete documentation on this topic. - ---------------- -Getting Started ---------------- - -A few things before we get started: - -- We'll refer to the Spack installation location via the environment - variable ``SPACK_ROOT``. You should point ``SPACK_ROOT`` at wherever - you have Spack installed. -- Add ``$SPACK_ROOT/bin`` to your ``PATH`` before you start. -- Make sure your ``EDITOR`` environment variable is set to your - preferred text editor. -- We'll be writing Python code as part of this tutorial. You can find - successive versions of the Python code in - ``$SPACK_ROOT/lib/spack/docs/tutorial/examples``. - -------------------------- -Creating the Package File -------------------------- - -We will use a separate package repository for the tutorial. Package -repositories allow you to separate sets of packages that take -precedence over one another. We will use the tutorial repo that ships -with Spack to avoid breaking the builtin Spack packages. - -.. code-block:: console - - $ spack repo add $SPACK_ROOT/var/spack/repos/tutorial/ - ==> Added repo with namespace 'tutorial'. - -Spack comes with a handy command to create a new package: ``spack create``. - -This command is given the location of a package's source code, downloads -the code, and sets up some basic packaging infrastructure for you. The -mpileaks source code can be found on GitHub, and here's what happens when -we run ``spack create`` on it: - -.. code-block:: console - - $ spack create -t generic https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz - ==> This looks like a URL for mpileaks - ==> Found 1 version of mpileaks: - - 1.0 https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz - - ==> How many would you like to checksum? (default is 1, q to abort) 1 - ==> Downloading... - ==> Fetching https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz - ############################################################################# 100.0% - ==> Checksummed 1 version of mpileaks - ==> Using specified package template: 'generic' - ==> Created template for mpileaks package - ==> Created package file: ~/spack/var/spack/repos/tutorial/packages/mpileaks/package.py - -Spack should spawn a text editor with this file: - -.. literalinclude:: tutorial/examples/0.package.py - :language: python - -Spack has created this file in -``$SPACK_ROOT/var/spack/repos/tutorial/packages/mpileaks/package.py``. Take a -moment to look over the file. There's a few placeholders that Spack has -created, which we'll fill in as part of this tutorial: - -- We'll document some information about this package in the comments. -- We'll fill in the dependency list for this package. -- We'll fill in some of the configuration arguments needed to build this - package. - -For the moment, exit your editor and let's see what happens when we try -to build this package: - -.. code-block:: console - - $ spack install mpileaks - ==> Installing mpileaks - ==> Searching for binary cache of mpileaks - ==> Warning: No Spack mirrors are currently configured - ==> No binary for mpileaks found: installing from source - ==> Fetching https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz - ############################################################################# 100.0% - ==> Staging archive: ~/spack/var/spack/stage/mpileaks-1.0-sv75n3u5ev6mljwcezisz3slooozbbxu/mpileaks-1.0.tar.gz - ==> Created stage in ~/spack/var/spack/stage/mpileaks-1.0-sv75n3u5ev6mljwcezisz3slooozbbxu - ==> No patches needed for mpileaks - ==> Building mpileaks [Package] - ==> Executing phase: 'install' - ==> Error: ProcessError: Command exited with status 2: - 'make' '-j16' - - 1 error found in build log: - 1 ==> Executing phase: 'install' - 2 ==> 'make' '-j16' - >> 3 make: *** No targets specified and no makefile found. Stop. - - See build log for details: - ~/spack/var/spack/stage/mpileaks-1.0-sv75n3u5ev6mljwcezisz3slooozbbxu/spack-build-out.txt - -This obviously didn't work; we need to fill in the package-specific -information. Specifically, Spack didn't try to build any of mpileaks' -dependencies, nor did it use the proper configure arguments. Let's start -fixing things. - ---------------------- -Package Documentation ---------------------- - -We can bring the ``package.py`` file back into our ``EDITOR`` with the -``spack edit`` command: - -.. code-block:: console - - $ spack edit mpileaks - -Let's remove some of the ``FIXME`` comments, add links to the mpileaks -homepage, and document what mpileaks does. I'm also going to cut out the -Copyright clause at this point to keep this tutorial document shorter, -but you shouldn't do that normally. The results of these changes can be -found in ``$SPACK_ROOT/lib/spack/docs/tutorial/examples/1.package.py`` -and are displayed below. Make these changes to your ``package.py``: - -.. literalinclude:: tutorial/examples/1.package.py - :lines: 6- - :language: python - -We've filled in the comment that describes what this package does and -added a link to its website. That won't help us build yet, but it will -allow Spack to provide some documentation on this package to other users: - -.. code-block:: console - - $ spack info mpileaks - Package: mpileaks - - Description: - Tool to detect and report MPI objects like MPI_Requests and - MPI_Datatypes. - - Homepage: https://github.com/LLNL/mpileaks - - Tags: - None - - Preferred version: - 1.0 https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz - - Safe versions: - 1.0 https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz - - Variants: - None - - Installation Phases: - install - - Build Dependencies: - None - - Link Dependencies: - None - - Run Dependencies: - None - - Virtual Packages: - None - -As we fill in more information about this package the ``spack info`` command -will become more informative. Now let's start making this package build. - ------------- -Dependencies ------------- - -The mpileaks package depends on three other packages: ``mpi``, -``adept-utils``, and ``callpath``. Let's add those via the -``depends_on`` command in our ``package.py`` (this version is in -``$SPACK_ROOT/lib/spack/docs/tutorial/examples/2.package.py``): - -.. literalinclude:: tutorial/examples/2.package.py - :lines: 6- - :language: python - -Now when we go to build mpileaks, Spack will fetch and build these -dependencies before building mpileaks. Note that the mpi dependency is a -different kind of beast than the adept-utils and callpath dependencies; -there is no mpi package available in Spack. Instead mpi is a *virtual -dependency*. Spack may satisfy that dependency by installing packages -such as ``openmpi`` or ``mvapich2``. See the :ref:`packaging-guide` for more -information on virtual dependencies. - -Now when we try to install this package, a lot more happens: - -.. code-block:: console - - $ spack install mpileaks - ... - ==> Successfully installed libdwarf from binary cache - [+] ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libdwarf-20180129-p4jeflorwlnkoq2vpuyocwrbcht2ayak - ==> Installing callpath - ==> Searching for binary cache of callpath - ==> Installing callpath from binary cache - ==> Fetching file:///mirror/build_cache/linux-ubuntu16.04-x86_64/gcc-5.4.0/callpath-1.0.4/linux-ubuntu16.04-x86_64-gcc-5.4.0-callpath-1.0.4-empvyxdkc4j4pwg7gznwhbiumruey66x.spack - ######################################################################## 100.0% - ==> Successfully installed callpath from binary cache - [+] ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/callpath-1.0.4-empvyxdkc4j4pwg7gznwhbiumruey66x - ==> Installing mpileaks - ==> Searching for binary cache of mpileaks - ==> No binary for mpileaks found: installing from source - ==> Using cached archive: ~/spack/var/spack/cache/mpileaks/mpileaks-1.0.tar.gz - ==> Staging archive: ~/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb/mpileaks-1.0.tar.gz - ==> Created stage in ~/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb - ==> No patches needed for mpileaks - ==> Building mpileaks [Package] - ==> Executing phase: 'install' - ==> Error: ProcessError: Command exited with status 2: - 'make' '-j16' - - 1 error found in build log: - 1 ==> Executing phase: 'install' - 2 ==> 'make' '-j16' - >> 3 make: *** No targets specified and no makefile found. Stop. - - See build log for details: - ~/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb/mpileaks-1.0/spack-build-out.txt - -Note that this command may take a while to run and produce more output if -you don't have an MPI already installed or configured in Spack. - -Now Spack has identified and made sure all of our dependencies have been -built. It found the ``openmpi`` package that will satisfy our ``mpi`` -dependency, and the ``callpath`` and ``adept-utils`` package to satisfy our -concrete dependencies. - ------------------------- -Debugging Package Builds ------------------------- - -Our ``mpileaks`` package is still not building. It may be obvious to -many of you that we never ran the configure script. Let's add a -call to ``configure()`` to the top of the install routine. The resulting -``package.py`` is in ``$SPACK_ROOT/lib/spack/docs/tutorial/examples/3.package.py``: - -.. literalinclude:: tutorial/examples/3.package.py - :lines: 6- - :language: python - -If we re-run we still get errors: - -.. code-block:: console - - $ spack install mpileaks - ... - ==> Installing mpileaks - ==> Searching for binary cache of mpileaks - ==> Finding buildcaches in /mirror/build_cache - ==> No binary for mpileaks found: installing from source - ==> Using cached archive: ~/spack/var/spack/cache/mpileaks/mpileaks-1.0.tar.gz - ==> Staging archive: ~/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb/mpileaks-1.0.tar.gz - ==> Created stage in ~/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb - ==> No patches needed for mpileaks - ==> Building mpileaks [Package] - ==> Executing phase: 'install' - ==> Error: ProcessError: Command exited with status 1: - './configure' - - 1 error found in build log: - 25 checking for ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.1.3-3 - njc4q5pqdpptq6jvqjrezkffwokv2sx/bin/mpicc... ~/spack/opt/spack/linux-ubuntu16.04- - x86_64/gcc-5.4.0/openmpi-3.1.3-3njc4q5pqdpptq6jvqjrezkffwokv2sx/bin/mpicc - 26 Checking whether ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.1 - .3-3njc4q5pqdpptq6jvqjrezkffwokv2sx/bin/mpicc responds to '-showme:compile'... no - 27 Checking whether ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.1 - .3-3njc4q5pqdpptq6jvqjrezkffwokv2sx/bin/mpicc responds to '-showme'... no - 28 Checking whether ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.1 - .3-3njc4q5pqdpptq6jvqjrezkffwokv2sx/bin/mpicc responds to '-compile-info'... no - 29 Checking whether ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.1 - .3-3njc4q5pqdpptq6jvqjrezkffwokv2sx/bin/mpicc responds to '-show'... no - 30 ./configure: line 4809: Echo: command not found - >> 31 configure: error: unable to locate adept-utils installation - - See build log for details: - ~/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb/mpileaks-1.0/spack-build-out.txt - -Again, the problem may be obvious. But let's pretend we're not -all experienced Autotools developers and use this opportunity to spend some -time debugging. We have a few options that can tell us about -what's going wrong: - -As per the error message, Spack has given us a ``spack-build-out.txt`` debug -log: - -.. code-block:: console - - ==> Executing phase: 'install' - ==> './configure' - checking metadata... no - checking installation directory variables... yes - checking for a BSD-compatible install... /usr/bin/install -c - checking whether build environment is sane... yes - checking for a thread-safe mkdir -p... /bin/mkdir -p - checking for gawk... gawk - checking whether make sets $(MAKE)... yes - checking for gcc... /home/spack1/spack/lib/spack/env/gcc/gcc - checking for C compiler default output file name... a.out - checking whether the C compiler works... yes - checking whether we are cross compiling... no - checking for suffix of executables... - checking for suffix of object files... o - checking whether we are using the GNU C compiler... yes - checking whether /home/spack1/spack/lib/spack/env/gcc/gcc accepts -g... yes - checking for /home/spack1/spack/lib/spack/env/gcc/gcc option to accept ISO C89... none needed - checking for style of include used by make... GNU - checking dependency style of /home/spack1/spack/lib/spack/env/gcc/gcc... gcc3 - checking whether /home/spack1/spack/lib/spack/env/gcc/gcc and cc understand -c and -o together... yes - checking whether we are using the GNU C++ compiler... yes - checking whether /home/spack1/spack/lib/spack/env/gcc/g++ accepts -g... yes - checking dependency style of /home/spack1/spack/lib/spack/env/gcc/g++... gcc3 - checking for /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc... /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc - Checking whether /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc responds to '-showme:compile'... yes - configure: error: unable to locate adept-utils installation - -This gives us the output from the build, and mpileaks isn't -finding its ``adept-utils`` package. Spack has -automatically added the include and library directories of -``adept-utils`` to the compiler's search path, but some packages like -mpileaks can sometimes be picky and still want things spelled out on -their command line. But let's continue to pretend we're not experienced -developers, and explore some other debugging paths: - -We can also enter the build area and try to manually run the build: - -.. code-block:: console - - $ spack build-env mpileaks bash - $ spack cd mpileaks - -The ``spack build-env`` command spawned a new shell that contains the same -environment that Spack used to build the mpileaks package (you can -substitute bash for your favorite shell). The ``spack cd`` command -changed our working dirctory to the last attempted build for mpileaks. -From here we can manually re-run the build: - -.. code-block:: console - - $ ./configure - checking metadata... no - checking installation directory variables... yes - checking for a BSD-compatible install... /usr/bin/install -c - checking whether build environment is sane... yes - checking for a thread-safe mkdir -p... /bin/mkdir -p - checking for gawk... gawk - checking whether make sets $(MAKE)... yes - checking for gcc... /home/spack1/spack/lib/spack/env/gcc/gcc - checking for C compiler default output file name... a.out - checking whether the C compiler works... yes - checking whether we are cross compiling... no - checking for suffix of executables... - checking for suffix of object files... o - checking whether we are using the GNU C compiler... yes - checking whether /home/spack1/spack/lib/spack/env/gcc/gcc accepts -g... yes - checking for /home/spack1/spack/lib/spack/env/gcc/gcc option to accept ISO C89... none needed - checking for style of include used by make... GNU - checking dependency style of /home/spack1/spack/lib/spack/env/gcc/gcc... gcc3 - checking whether /home/spack1/spack/lib/spack/env/gcc/gcc and cc understand -c and -o together... yes - checking whether we are using the GNU C++ compiler... yes - checking whether /home/spack1/spack/lib/spack/env/gcc/g++ accepts -g... yes - checking dependency style of /home/spack1/spack/lib/spack/env/gcc/g++... gcc3 - checking for /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc... /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc - Checking whether /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc responds to '-showme:compile'... yes - configure: error: unable to locate adept-utils installation - -We're seeing the same error, but now we're in a shell where we can run -the command ourselves and debug as needed. We could, for example, run -``./configure --help`` to see what options we can use to specify -dependencies. - -We can use the ``exit`` command to leave the shell spawned by ``spack -build-env``. - ------------------------------- -Specifying Configure Arguments ------------------------------- - -Let's add the configure arguments to the mpileaks' ``package.py``. This -version can be found in -``$SPACK_ROOT/lib/spack/docs/tutorial/examples/4.package.py``: - -.. literalinclude:: tutorial/examples/4.package.py - :lines: 6- - :language: python - -This is all we need for a working mpileaks package! If we install now we'll see: - -.. code-block:: console - - $ spack install mpileaks - ... - ==> Installing mpileaks - ==> Searching for binary cache of mpileaks - ==> Finding buildcaches in /mirror/build_cache - ==> No binary for mpileaks found: installing from source - ==> Using cached archive: ~/spack/var/spack/cache/mpileaks/mpileaks-1.0.tar.gz - ==> Staging archive: ~/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb/mpileaks-1.0.tar.gz - ==> Created stage in ~/spack/var/spack/stage/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb - ==> No patches needed for mpileaks - ==> Building mpileaks [Package] - ==> Executing phase: 'install' - ==> Successfully installed mpileaks - Fetch: 0.00s. Build: 9.41s. Total: 9.41s. - [+] ~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/mpileaks-1.0-csoikctsalli4cdkkdk377gprkc472rb - -There are some special circumstances in this package that are worth highlighting. -Normally, Spack would have automatically detected that mpileaks was an -Autotools-based package when we ran ``spack create`` and made it an ``AutoToolsPackage`` -class (except we added the ``-t generic`` option to skip this). Instead of -a full install routine we would have just written: - -.. code-block:: python - - def configure_args(self): - return [ - '--with-adept-utils={0}'.format(self.spec['adept-utils'].prefix), - '--with-callpath={0}'.format(self.spec['callpath'].prefix) - ] - -Similarly, if this had been a CMake-based package we -would have been filling in a ``cmake_args`` function instead of -``configure_args``. There are similar default package types for -many build environments that will be discussed later in the tutorial. - --------- -Variants --------- - -We have a successful mpileaks build, but let's take some time to improve -it. ``mpileaks`` has a build-time option to truncate parts of the stack -that it walks. Let's add a variant to allow users to set this when they -build mpileaks with Spack. - -To do this, we'll add a variant to our package, as per the following (see -``$SPACK_ROOT/lib/spack/docs/tutorial/examples/5.package.py``): - -.. literalinclude:: tutorial/examples/5.package.py - :lines: 6- - :language: python - -We've added the variant ``stackstart``, and given it a default value of -``0``. If we install now we can see the stackstart variant added to the -configure line (output truncated for length): - -.. code-block:: console - - $ spack install --verbose mpileaks stackstart=4 - ... - ==> Installing mpileaks - ==> Searching for binary cache of mpileaks - ==> Finding buildcaches in /mirror/build_cache - ==> No binary for mpileaks found: installing from source - ==> Using cached archive: ~/spack/var/spack/cache/mpileaks/mpileaks-1.0.tar.gz - ==> Staging archive: ~/spack/var/spack/stage/mpileaks-1.0-meufjojkxve3l7rci2mbud3faidgplto/mpileaks-1.0.tar.gz - ==> Created stage in ~/spack/var/spack/stage/mpileaks-1.0-meufjojkxve3l7rci2mbud3faidgplto - ==> No patches needed for mpileaks - ==> Building mpileaks [Package] - ==> Executing phase: 'install' - ==> './configure' '--prefix=~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/mpileaks-1.0-meufjojkxve3l7rci2mbud3faidgplto' '--with-adept-utils=~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/adept-utils-1.0.1-7tippnvo5g76wpijk7x5kwfpr3iqiaen' '--with-callpath=~/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/callpath-1.0.4-empvyxdkc4j4pwg7gznwhbiumruey66x' '--with-stack-start-c=4' '--with-stack-start-fortran=4' - ---------------- -The Spec Object ---------------- - -This tutorial has glossed over a few important features, which weren't -too relevant for mpileaks but may be useful for other packages. There -were several places we reference the ``self.spec`` object. This is a -powerful class for querying information about what we're building. For -example, you could use the spec to query information about how a -package's dependencies were built, or what compiler was being used, or -what version of a package is being installed. Full documentation can be -found in the :ref:`packaging-guide`, but here's some quick snippets with -common queries: - -- Am I building ``mpileaks`` version ``1.1`` or greater? - -.. code-block:: python - - if self.spec.satisfies('@1.1:'): - # Do things needed for 1.1+ - -- Is ``openmpi`` the MPI I'm building with? - -.. code-block:: python - - if self.spec['mpi'].name == 'openmpi': - # Do openmpi things - -- Am I building with ``gcc`` version less than ``5.0.0``: - -.. code-block:: python - - if self.spec.satisfies('%gcc@:5.0.0'): - # Add arguments specific to gcc's earlier than 5.0.0 - -- Am I building with the ``debug`` variant: - -.. code-block:: python - - if self.spec.satisfies('+debug'): - # Add -g option to configure flags - -- Is my ``dyninst`` dependency greater than version ``8.0``? - -.. code-block:: python - - if self.spec['dyninst'].satisfies('@8.0:'): - # Use newest dyninst options - -More examples can be found in the thousands of packages already added to -Spack in ``$SPACK_ROOT/var/spack/repos/builtin/packages``. - -Good Luck! - -To ensure that future sections of the tutorial run properly, please -uninstall mpileaks and remove the tutorial repo from your -configuration. - -.. code-block:: console - - $ spack uninstall -ay mpileaks - $ spack repo remove tutorial - $ rm -rf $SPACK_ROOT/var/spack/repos/tutorial/packages/mpileaks |
