summaryrefslogtreecommitdiff
path: root/lib
diff options
context:
space:
mode:
authorMassimiliano Culpo <massimiliano.culpo@gmail.com>2020-10-30 21:17:15 +0100
committerGitHub <noreply@github.com>2020-10-30 21:17:15 +0100
commitc4aa5cb5bcc72bf5f3ba4e78338cca35394043f2 (patch)
tree7abddacaa371e4d36aa22d1d9040dcbbe2143483 /lib
parent33c3c3c700cf0acd458389526974945e62fe895e (diff)
downloadspack-c4aa5cb5bcc72bf5f3ba4e78338cca35394043f2.tar.gz
spack-c4aa5cb5bcc72bf5f3ba4e78338cca35394043f2.tar.bz2
spack-c4aa5cb5bcc72bf5f3ba4e78338cca35394043f2.tar.xz
spack-c4aa5cb5bcc72bf5f3ba4e78338cca35394043f2.zip
Update documentation on containers (#19631)
fixes #15183 - Moved the container related content from workflows.rst into containers.rst - Deleted the docker_for_developers.rst file, since it describes an outdated procedure Co-authored-by: Axel Huebl <a.huebl@hzdr.de> Co-authored-by: Omar Padron <omar.padron@kitware.com>
Diffstat (limited to 'lib')
-rw-r--r--lib/spack/docs/containers.rst61
-rw-r--r--lib/spack/docs/docker_for_developers.rst41
-rw-r--r--lib/spack/docs/index.rst1
-rw-r--r--lib/spack/docs/workflows.rst165
4 files changed, 61 insertions, 207 deletions
diff --git a/lib/spack/docs/containers.rst b/lib/spack/docs/containers.rst
index 78e79286d2..2590fec8b6 100644
--- a/lib/spack/docs/containers.rst
+++ b/lib/spack/docs/containers.rst
@@ -79,6 +79,13 @@ Environments:
ENTRYPOINT ["/bin/bash", "--rcfile", "/etc/profile", "-l"]
+In order to build and run the image, execute:
+
+.. code-block:: bash
+
+ $ spack containerize > Dockerfile
+ $ docker build -t myimage .
+ $ docker run -it myimage
The bits that make this automation possible are discussed in details
below. All the images generated in this way will be based on
@@ -305,3 +312,57 @@ following ``Dockerfile``:
Spack can also produce Singularity definition files to build the image. The
minimum version of Singularity required to build a SIF (Singularity Image Format)
from them is ``3.5.3``.
+
+--------------
+Best Practices
+--------------
+
+^^^
+MPI
+^^^
+Due to the dependency on Fortran for OpenMPI, which is the spack default
+implementation, consider adding ``gfortran`` to the ``apt-get install`` list.
+
+Recent versions of OpenMPI will require you to pass ``--allow-run-as-root``
+to your ``mpirun`` calls if started as root user inside Docker.
+
+For execution on HPC clusters, it can be helpful to import the docker
+image into Singularity in order to start a program with an *external*
+MPI. Otherwise, also add ``openssh-server`` to the ``apt-get install`` list.
+
+^^^^
+CUDA
+^^^^
+Starting from CUDA 9.0, Nvidia provides minimal CUDA images based on
+Ubuntu. Please see `their instructions <https://hub.docker.com/r/nvidia/cuda/>`_.
+Avoid double-installing CUDA by adding, e.g.
+
+.. code-block:: yaml
+
+ packages:
+ cuda:
+ externals:
+ - spec: "cuda@9.0.176%gcc@5.4.0 arch=linux-ubuntu16-x86_64"
+ prefix: /usr/local/cuda
+ buildable: False
+
+to your ``spack.yaml``.
+
+Users will either need ``nvidia-docker`` or e.g. Singularity to *execute*
+device kernels.
+
+^^^^^^^^^^^^^^^^^^^^^^^^^
+Docker on Windows and OSX
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+On Mac OS and Windows, docker runs on a hypervisor that is not allocated much
+memory by default, and some spack packages may fail to build due to lack of
+memory. To work around this issue, consider configuring your docker installation
+to use more of your host memory. In some cases, you can also ease the memory
+pressure on parallel builds by limiting the parallelism in your config.yaml.
+
+.. code-block:: yaml
+
+ config:
+ build_jobs: 2
+
diff --git a/lib/spack/docs/docker_for_developers.rst b/lib/spack/docs/docker_for_developers.rst
deleted file mode 100644
index 2b489cdec4..0000000000
--- a/lib/spack/docs/docker_for_developers.rst
+++ /dev/null
@@ -1,41 +0,0 @@
-.. Copyright 2013-2020 Lawrence Livermore National Security, LLC and other
- Spack Project Developers. See the top-level COPYRIGHT file for details.
-
- SPDX-License-Identifier: (Apache-2.0 OR MIT)
-
-.. _docker_for_developers:
-
-=====================
-Docker for Developers
-=====================
-
-This guide is intended for people who want to use our prepared docker
-environments to work on developing Spack or working on spack packages. It is
-meant to serve as the companion documentation for the :ref:`packaging-guide`.
-
---------
-Overview
---------
-
-To get started, all you need is the latest version of ``docker``.
-
-.. code-block:: console
-
- $ cd share/spack/docker
- $ source config/ubuntu.bash
- $ ./run-image.sh
-
-This command should drop you into an interactive shell where you can run spack
-within an isolated docker container running ubuntu. The copy of spack being
-used should be tied to the working copy of your cloned git repo, so any changes
-you make should be immediately reflected in the running docker container. Feel
-free to add or modify any packages or to hack on spack, itself. Your contained
-copy of spack should immediately reflect all changes.
-
-To work within a container running a different linux distro, source one of the
-other environment files under ``config``.
-
-.. code-block:: console
-
- $ source config/fedora.bash
- $ ./run-image.sh
diff --git a/lib/spack/docs/index.rst b/lib/spack/docs/index.rst
index 8170f152a4..39fee81387 100644
--- a/lib/spack/docs/index.rst
+++ b/lib/spack/docs/index.rst
@@ -85,7 +85,6 @@ or refer to the full manual below.
packaging_guide
build_systems
developer_guide
- docker_for_developers
.. toctree::
:maxdepth: 2
diff --git a/lib/spack/docs/workflows.rst b/lib/spack/docs/workflows.rst
index 4a56563283..4d3a97012c 100644
--- a/lib/spack/docs/workflows.rst
+++ b/lib/spack/docs/workflows.rst
@@ -1034,171 +1034,6 @@ The main points that are implemented below:
- make -j 2
- make test
-.. _workflow_create_docker_image:
-
------------------------------------
-Using Spack to Create Docker Images
------------------------------------
-
-Spack can be the ideal tool to set up images for Docker (and Singularity).
-
-An example ``Dockerfile`` is given below, downloading the latest spack
-version.
-
-The following functionality is prepared:
-
-#. Base image: the example starts from a minimal ubuntu.
-
-#. Pre-install the spack dependencies.
- Package installs are followed by a clean-up of the system package index,
- to avoid outdated information and it saves space.
-
-#. Install spack in ``/usr/local``.
- Add ``setup-env.sh`` to profile scripts, so commands in *login* shells
- can use the whole spack functionality, including modules.
-
-#. Install an example package (``tar``).
- As with system package managers above, ``spack install`` commands should be
- concatenated with a ``&& spack clean -a`` in order to keep image sizes small.
-
-#. Add a startup hook to an *interactive login shell* so spack modules will be
- usable.
-
-In order to build and run the image, execute:
-
-.. code-block:: bash
-
- docker build -t spack .
- docker run -it spack
-
-.. code-block:: docker
-
- FROM ubuntu:16.04
- MAINTAINER Your Name <someone@example.com>
-
- # general environment for docker
- ENV DEBIAN_FRONTEND=noninteractive \
- SPACK_ROOT=/usr/local
-
- # install minimal spack dependencies
- RUN apt-get update \
- && apt-get install -y --no-install-recommends \
- autoconf \
- build-essential \
- ca-certificates \
- coreutils \
- curl \
- environment-modules \
- git \
- python \
- unzip \
- vim \
- && rm -rf /var/lib/apt/lists/*
-
- # load spack environment on login
- RUN echo "source $SPACK_ROOT/share/spack/setup-env.sh" \
- > /etc/profile.d/spack.sh
-
- # spack settings
- # note: if you wish to change default settings, add files alongside
- # the Dockerfile with your desired settings. Then uncomment this line
- #COPY packages.yaml modules.yaml $SPACK_ROOT/etc/spack/
-
- # install spack
- RUN curl -s -L https://api.github.com/repos/spack/spack/tarball \
- | tar xzC $SPACK_ROOT --strip 1
- # note: at this point one could also run ``spack bootstrap`` to avoid
- # parts of the long apt-get install list above
-
- # install software
- RUN spack install tar \
- && spack clean -a
-
- # need the executables from a package already during image build?
- #RUN /bin/bash -l -c ' \
- # spack load tar \
- # && which tar'
-
- # image run hook: the -l will make sure /etc/profile environments are loaded
- CMD /bin/bash -l
-
-^^^^^^^^^^^^^^
-Best Practices
-^^^^^^^^^^^^^^
-
-"""
-MPI
-"""
-Due to the dependency on Fortran for OpenMPI, which is the spack default
-implementation, consider adding ``gfortran`` to the ``apt-get install`` list.
-
-Recent versions of OpenMPI will require you to pass ``--allow-run-as-root``
-to your ``mpirun`` calls if started as root user inside Docker.
-
-For execution on HPC clusters, it can be helpful to import the docker
-image into Singularity in order to start a program with an *external*
-MPI. Otherwise, also add ``openssh-server`` to the ``apt-get install`` list.
-
-""""
-CUDA
-""""
-Starting from CUDA 9.0, Nvidia provides minimal CUDA images based on
-Ubuntu.
-Please see `their instructions <https://hub.docker.com/r/nvidia/cuda/>`_.
-Avoid double-installing CUDA by adding, e.g.
-
-.. code-block:: yaml
-
- packages:
- cuda:
- externals:
- - spec: "cuda@9.0.176%gcc@5.4.0 arch=linux-ubuntu16-x86_64"
- prefix: /usr/local/cuda
- buildable: False
-
-to your ``packages.yaml``.
-Then ``COPY`` in that file into the image as in the example above.
-
-Users will either need ``nvidia-docker`` or e.g. Singularity to *execute*
-device kernels.
-
-"""""""""""
-Singularity
-"""""""""""
-Importing and running the image created above into
-`Singularity <http://singularity.lbl.gov/>`_ works like a charm.
-Just use the `docker bootstraping mechanism <http://singularity.lbl.gov/quickstart#bootstrap-recipes>`_:
-
-.. code-block:: none
-
- Bootstrap: docker
- From: registry/user/image:tag
-
- %runscript
- exec /bin/bash -l
-
-""""""""""""""""""""""
-Docker for Development
-""""""""""""""""""""""
-
-For examples of how we use docker in development, see
-:ref:`docker_for_developers`.
-
-"""""""""""""""""""""""""
-Docker on Windows and OSX
-"""""""""""""""""""""""""
-
-On Mac OS and Windows, docker runs on a hypervisor that is not allocated much
-memory by default, and some spack packages may fail to build due to lack of
-memory. To work around this issue, consider configuring your docker installation
-to use more of your host memory. In some cases, you can also ease the memory
-pressure on parallel builds by limiting the parallelism in your config.yaml.
-
-.. code-block:: yaml
-
- config:
- build_jobs: 2
-
------------------
Upstream Bug Fixes
------------------