From e17765db39deb70b9a140ff197339d8cba417f6b Mon Sep 17 00:00:00 2001 From: Axel Huebl Date: Fri, 13 Oct 2017 21:17:20 +0200 Subject: Docker Workflow (#5582) This adds a workflow section on how to use spack on Docker. It provides an example on the best-practices I collected over the last months and circumvents the common pitfalls I tapped in. Works with MPI, CUDA, Modules, execution as root, etc. Background: Developed initially for PIConGPU. --- lib/spack/docs/workflows.rst | 143 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 143 insertions(+) (limited to 'lib') diff --git a/lib/spack/docs/workflows.rst b/lib/spack/docs/workflows.rst index 10843d302b..b29feccea4 100644 --- a/lib/spack/docs/workflows.rst +++ b/lib/spack/docs/workflows.rst @@ -1106,6 +1106,149 @@ The main points that are implemented below: - make -j 2 - make test +----------------------------------- +Using Spack to Create Docker Images +----------------------------------- + +Spack can be the ideal tool to set up images for Docker (and Singularity). + +An example ``Dockerfile`` is given below, downloading the latest spack +version. + +The following functionality is prepared: + +#. Base image: the example starts from a minimal ubuntu. + +#. Installing as root: docker images are usually set up as root. + Since some autotools scripts might complain about this being unsafe, we set + ``FORCE_UNSAFE_CONFIGURE=1`` to avoid configure errors. + +#. Pre-install the spack dependencies, including modules from the packages. + This avoids needing to build those from scratch via ``spack bootstrap``. + Package installs are followed by a clean-up of the system package index, + to avoid outdated information and it saves space. + +#. Install spack in ``/usr/local``. + Add ``setup-env.sh`` to profile scripts, so commands in *login* shells + can use the whole spack functionality, including modules. + +#. Install an example package (``tar``). + As with system package managers above, ``spack install`` commands should be + concatenated with a ``&& spack clean -a`` in order to keep image sizes small. + +#. Add a startup hook to an *interactive login shell* so spack modules will be + usable. + +In order to build and run the image, execute: + +.. code-block:: bash + + docker build -t spack . + docker run -it spack + +.. code-block:: docker + + FROM ubuntu:16.04 + MAINTAINER Your Name + + # general environment for docker + ENV DEBIAN_FRONTEND=noninteractive \ + SPACK_ROOT=/usr/local \ + FORCE_UNSAFE_CONFIGURE=1 + + # install minimal spack depedencies + RUN apt-get update \ + && apt-get install -y --no-install-recommends \ + autoconf \ + build-essential \ + ca-certificates \ + coreutils \ + curl \ + environment-modules \ + git \ + python \ + unzip \ + vim \ + && rm -rf /var/lib/apt/lists/* + + # load spack environment on login + RUN echo "source $SPACK_ROOT/share/spack/setup-env.sh" \ + > /etc/profile.d/spack.sh + + # spack settings + # note: if you wish to change default settings, add files alongside + # the Dockerfile with your desired settings. Then uncomment this line + #COPY packages.yaml modules.yaml $SPACK_ROOT/etc/spack/ + + # install spack + RUN curl -s -L https://api.github.com/repos/llnl/spack/tarball \ + | tar xzC $SPACK_ROOT --strip 1 + # note: at this point one could also run ``spack bootstrap`` to avoid + # parts of the long apt-get install list above + + # install software + RUN spack install tar \ + && spack clean -a + + # need the modules already during image build? + #RUN /bin/bash -l -c ' \ + # spack load tar \ + # && which tar' + + # image run hook: the -l will make sure /etc/profile environments are loaded + CMD /bin/bash -l + +^^^^^^^^^^^^^^ +Best Practices +^^^^^^^^^^^^^^ + +""" +MPI +""" +Due to the dependency on Fortran for OpenMPI, which is the spack default +implementation, consider adding ``gfortran`` to the ``apt-get install`` list. + +For execution on HPC clusters, it can be helpful to import the docker +image into Singularity in order to start a program with an *external* +MPI. Otherwise, also add ``openssh-server`` to the ``apt-get install`` list. + +"""" +CUDA +"""" +Starting from CUDA 9.0, Nvidia provides minimal CUDA images based on +Ubuntu. +Please see `their instructions `_. +Avoid double-installing CUDA by adding, e.g. + +.. code-block:: yaml + + packages: + cuda: + paths: + cuda@9.0.176%gcc@5.4.0 arch=linux-ubuntu16-x86_64: /usr/local/cuda + buildable: False + +to your ``packages.yaml``. +Then ``COPY`` in that file into the image as in the example above. + +Users will either need ``nvidia-docker`` or e.g. Singularity to *execute* +device kernels. + +""""""""""" +Singularity +""""""""""" +Importing and running the image created above into +`Singularity `_ works like a charm. +Just use the `docker bootstraping mechanism `_: + +.. code-block:: none + + Bootstrap: docker + From: registry/user/image:tag + + %runscript + exec /bin/bash -l + ------------------ Upstream Bug Fixes ------------------ -- cgit v1.2.3-70-g09d2