diff options
| -rw-r--r-- | lib/spack/docs/basic_usage.rst | 89 | ||||
| -rw-r--r-- | lib/spack/docs/build_systems/pythonpackage.rst | 9 | ||||
| -rw-r--r-- | lib/spack/docs/build_systems/rpackage.rst | 8 | ||||
| -rw-r--r-- | lib/spack/docs/packaging_guide.rst | 16 |
4 files changed, 78 insertions, 44 deletions
diff --git a/lib/spack/docs/basic_usage.rst b/lib/spack/docs/basic_usage.rst index b26429b248..15e1a23bfc 100644 --- a/lib/spack/docs/basic_usage.rst +++ b/lib/spack/docs/basic_usage.rst @@ -1672,9 +1672,13 @@ own install prefix. However, certain packages are typically installed `Python <https://www.python.org>`_ packages are typically installed in the ``$prefix/lib/python-2.7/site-packages`` directory. -Spack has support for this type of installation as well. In Spack, -a package that can live inside the prefix of another package is called -an *extension*. Suppose you have Python installed like so: +In Spack, installation prefixes are immutable, so this type of installation +is not directly supported. However, it is possible to create views that +allow you to merge install prefixes of multiple packages into a single new prefix. +Views are a convenient way to get a more traditional filesystem structure. +Using *extensions*, you can ensure that Python packages always share the +same prefix in the view as Python itself. Suppose you have +Python installed like so: .. code-block:: console @@ -1712,8 +1716,6 @@ You can find extensions for your Python installation like this: py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1 py-matplotlib@1.4.2 py-pyparsing@2.0.3 py-six@1.9.0 - ==> None activated. - The extensions are a subset of what's returned by ``spack list``, and they are packages like any other. They are installed into their own prefixes, and you can see this with ``spack find --paths``: @@ -1741,32 +1743,72 @@ directly when you run ``python``: ImportError: No module named numpy >>> -^^^^^^^^^^^^^^^^ -Using Extensions -^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Using Extensions in Environments +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The recommended way of working with extensions such as ``py-numpy`` +above is through :ref:`Environments <environments>`. For example, +the following creates an environment in the current working directory +with a filesystem view in the ``./view`` directory: + +.. code-block:: console + + $ spack env create --with-view view --dir . + $ spack -e . add py-numpy + $ spack -e . concretize + $ spack -e . install + +We recommend environments for two reasons. Firstly, environments +can be activated (requires :ref:`shell-support`): + +.. code-block:: console + + $ spack env activate . + +which sets all the right environment variables such as ``PATH`` and +``PYTHONPATH``. This ensures that + +.. code-block:: console + + $ python + >>> import numpy + +works. Secondly, even without shell support, the view ensures +that Python can locate its extensions: + +.. code-block:: console + + $ ./view/bin/python + >>> import numpy + +See :ref:`environments` for a more in-depth description of Spack +environments and customizations to views. -There are multiple ways to get ``numpy`` working in Python. The first is -to use :ref:`shell-support`. You can simply ``load`` the extension, -and it will be added to the ``PYTHONPATH`` in your current shell, and -Python itself will be available in the ``PATH``: +^^^^^^^^^^^^^^^^^^^^ +Using ``spack load`` +^^^^^^^^^^^^^^^^^^^^ + +A more traditional way of using Spack and extensions is ``spack load`` +(requires :ref:`shell-support`). This will add the extension to ``PYTHONPATH`` +in your current shell, and Python itself will be available in the ``PATH``: .. code-block:: console $ spack load py-numpy + $ python + >>> import numpy -Now ``import numpy`` will succeed for as long as you keep your current -session open. The loaded packages can be checked using ``spack find --loaded`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Loading Extensions via Modules ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Instead of using Spack's environment modification capabilities through -the ``spack load`` command, you can load numpy through your -environment modules (using ``environment-modules`` or ``lmod``). This -will also add the extension to the ``PYTHONPATH`` in your current -shell. +Apart from ``spack env activate`` and ``spack load``, you can load numpy +through your environment modules (using ``environment-modules`` or +``lmod``). This will also add the extension to the ``PYTHONPATH`` in +your current shell. .. code-block:: console @@ -1776,15 +1818,6 @@ If you do not know the name of the specific numpy module you wish to load, you can use the ``spack module tcl|lmod loads`` command to get the name of the module from the Spack spec. -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Extensions in an Environment -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Another way to use extensions is to create a view, which merges the -python installation along with the extensions into a single prefix. -See :ref:`environments` for a more in-depth description -of environment views. - ----------------------- Filesystem requirements ----------------------- diff --git a/lib/spack/docs/build_systems/pythonpackage.rst b/lib/spack/docs/build_systems/pythonpackage.rst index 53d0f09167..9be95d1fb9 100644 --- a/lib/spack/docs/build_systems/pythonpackage.rst +++ b/lib/spack/docs/build_systems/pythonpackage.rst @@ -724,10 +724,9 @@ extends vs. depends_on This is very similar to the naming dilemma above, with a slight twist. As mentioned in the :ref:`Packaging Guide <packaging_extensions>`, -``extends`` and ``depends_on`` are very similar, but ``extends`` adds -the ability to *activate* the package. Activation involves symlinking -everything in the installation prefix of the package to the installation -prefix of Python. This allows the user to import a Python module without +``extends`` and ``depends_on`` are very similar, but ``extends`` ensures +that the extension and extendee share the same prefix in views. +This allows the user to import a Python module without having to add that module to ``PYTHONPATH``. When deciding between ``extends`` and ``depends_on``, the best rule of @@ -735,7 +734,7 @@ thumb is to check the installation prefix. If Python libraries are installed to ``<prefix>/lib/pythonX.Y/site-packages``, then you should use ``extends``. If Python libraries are installed elsewhere or the only files that get installed reside in ``<prefix>/bin``, then -don't use ``extends``, as symlinking the package wouldn't be useful. +don't use ``extends``. ^^^^^^^^^^^^^^^^^^^^^ Alternatives to Spack diff --git a/lib/spack/docs/build_systems/rpackage.rst b/lib/spack/docs/build_systems/rpackage.rst index ebf2270e8e..567b79a863 100644 --- a/lib/spack/docs/build_systems/rpackage.rst +++ b/lib/spack/docs/build_systems/rpackage.rst @@ -193,10 +193,10 @@ Build system dependencies As an extension of the R ecosystem, your package will obviously depend on R to build and run. Normally, we would use ``depends_on`` to express -this, but for R packages, we use ``extends``. ``extends`` is similar to -``depends_on``, but adds an additional feature: the ability to "activate" -the package by symlinking it to the R installation directory. Since -every R package needs this, the ``RPackage`` base class contains: +this, but for R packages, we use ``extends``. This implies a special +dependency on R, which is used to set environment variables such as +``R_LIBS`` uniformly. Since every R package needs this, the ``RPackage`` +base class contains: .. code-block:: python diff --git a/lib/spack/docs/packaging_guide.rst b/lib/spack/docs/packaging_guide.rst index fb1d5edcab..5b0e556b34 100644 --- a/lib/spack/docs/packaging_guide.rst +++ b/lib/spack/docs/packaging_guide.rst @@ -2634,9 +2634,12 @@ extendable package: extends('python') ... -Now, the ``py-numpy`` package can be used as an argument to ``spack -activate``. When it is activated, all the files in its prefix will be -symbolically linked into the prefix of the python package. +This accomplishes a few things. Firstly, the Python package can set special +variables such as ``PYTHONPATH`` for all extensions when the run or build +environment is set up. Secondly, filesystem views can ensure that extensions +are put in the same prefix as their extendee. This ensures that Python in +a view can always locate its Python packages, even without environment +variables set. A package can only extend one other package at a time. To support packages that may extend one of a list of other packages, Spack supports multiple @@ -2684,9 +2687,8 @@ variant(s) are selected. This may be accomplished with conditional ... Sometimes, certain files in one package will conflict with those in -another, which means they cannot both be activated (symlinked) at the -same time. In this case, you can tell Spack to ignore those files -when it does the activation: +another, which means they cannot both be used in a view at the +same time. In this case, you can tell Spack to ignore those files: .. code-block:: python @@ -2698,7 +2700,7 @@ when it does the activation: ... The code above will prevent everything in the ``$prefix/bin/`` directory -from being linked in at activation time. +from being linked in a view. .. note:: |