summaryrefslogtreecommitdiff
path: root/lib
diff options
context:
space:
mode:
authorHarmen Stoppels <harmenstoppels@gmail.com>2022-11-11 13:24:17 +0100
committerMassimiliano Culpo <massimiliano.culpo@gmail.com>2022-11-11 16:31:25 +0100
commitfc7a16e77ebd51f9a86a726b412af9f2a21aa8c4 (patch)
tree95b0fdf8768204578f280facd50f5cceb586a896 /lib
parente633e57297ad6ddf95b7c647454c95f4c834f427 (diff)
downloadspack-fc7a16e77ebd51f9a86a726b412af9f2a21aa8c4.tar.gz
spack-fc7a16e77ebd51f9a86a726b412af9f2a21aa8c4.tar.bz2
spack-fc7a16e77ebd51f9a86a726b412af9f2a21aa8c4.tar.xz
spack-fc7a16e77ebd51f9a86a726b412af9f2a21aa8c4.zip
docs: updates related to extensions (#33837)
Diffstat (limited to 'lib')
-rw-r--r--lib/spack/docs/basic_usage.rst89
-rw-r--r--lib/spack/docs/build_systems/pythonpackage.rst9
-rw-r--r--lib/spack/docs/build_systems/rpackage.rst8
-rw-r--r--lib/spack/docs/packaging_guide.rst16
4 files changed, 78 insertions, 44 deletions
diff --git a/lib/spack/docs/basic_usage.rst b/lib/spack/docs/basic_usage.rst
index b26429b248..15e1a23bfc 100644
--- a/lib/spack/docs/basic_usage.rst
+++ b/lib/spack/docs/basic_usage.rst
@@ -1672,9 +1672,13 @@ own install prefix. However, certain packages are typically installed
`Python <https://www.python.org>`_ packages are typically installed in the
``$prefix/lib/python-2.7/site-packages`` directory.
-Spack has support for this type of installation as well. In Spack,
-a package that can live inside the prefix of another package is called
-an *extension*. Suppose you have Python installed like so:
+In Spack, installation prefixes are immutable, so this type of installation
+is not directly supported. However, it is possible to create views that
+allow you to merge install prefixes of multiple packages into a single new prefix.
+Views are a convenient way to get a more traditional filesystem structure.
+Using *extensions*, you can ensure that Python packages always share the
+same prefix in the view as Python itself. Suppose you have
+Python installed like so:
.. code-block:: console
@@ -1712,8 +1716,6 @@ You can find extensions for your Python installation like this:
py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1
py-matplotlib@1.4.2 py-pyparsing@2.0.3 py-six@1.9.0
- ==> None activated.
-
The extensions are a subset of what's returned by ``spack list``, and
they are packages like any other. They are installed into their own
prefixes, and you can see this with ``spack find --paths``:
@@ -1741,32 +1743,72 @@ directly when you run ``python``:
ImportError: No module named numpy
>>>
-^^^^^^^^^^^^^^^^
-Using Extensions
-^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Using Extensions in Environments
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The recommended way of working with extensions such as ``py-numpy``
+above is through :ref:`Environments <environments>`. For example,
+the following creates an environment in the current working directory
+with a filesystem view in the ``./view`` directory:
+
+.. code-block:: console
+
+ $ spack env create --with-view view --dir .
+ $ spack -e . add py-numpy
+ $ spack -e . concretize
+ $ spack -e . install
+
+We recommend environments for two reasons. Firstly, environments
+can be activated (requires :ref:`shell-support`):
+
+.. code-block:: console
+
+ $ spack env activate .
+
+which sets all the right environment variables such as ``PATH`` and
+``PYTHONPATH``. This ensures that
+
+.. code-block:: console
+
+ $ python
+ >>> import numpy
+
+works. Secondly, even without shell support, the view ensures
+that Python can locate its extensions:
+
+.. code-block:: console
+
+ $ ./view/bin/python
+ >>> import numpy
+
+See :ref:`environments` for a more in-depth description of Spack
+environments and customizations to views.
-There are multiple ways to get ``numpy`` working in Python. The first is
-to use :ref:`shell-support`. You can simply ``load`` the extension,
-and it will be added to the ``PYTHONPATH`` in your current shell, and
-Python itself will be available in the ``PATH``:
+^^^^^^^^^^^^^^^^^^^^
+Using ``spack load``
+^^^^^^^^^^^^^^^^^^^^
+
+A more traditional way of using Spack and extensions is ``spack load``
+(requires :ref:`shell-support`). This will add the extension to ``PYTHONPATH``
+in your current shell, and Python itself will be available in the ``PATH``:
.. code-block:: console
$ spack load py-numpy
+ $ python
+ >>> import numpy
-Now ``import numpy`` will succeed for as long as you keep your current
-session open.
The loaded packages can be checked using ``spack find --loaded``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Loading Extensions via Modules
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Instead of using Spack's environment modification capabilities through
-the ``spack load`` command, you can load numpy through your
-environment modules (using ``environment-modules`` or ``lmod``). This
-will also add the extension to the ``PYTHONPATH`` in your current
-shell.
+Apart from ``spack env activate`` and ``spack load``, you can load numpy
+through your environment modules (using ``environment-modules`` or
+``lmod``). This will also add the extension to the ``PYTHONPATH`` in
+your current shell.
.. code-block:: console
@@ -1776,15 +1818,6 @@ If you do not know the name of the specific numpy module you wish to
load, you can use the ``spack module tcl|lmod loads`` command to get
the name of the module from the Spack spec.
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Extensions in an Environment
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Another way to use extensions is to create a view, which merges the
-python installation along with the extensions into a single prefix.
-See :ref:`environments` for a more in-depth description
-of environment views.
-
-----------------------
Filesystem requirements
-----------------------
diff --git a/lib/spack/docs/build_systems/pythonpackage.rst b/lib/spack/docs/build_systems/pythonpackage.rst
index 53d0f09167..9be95d1fb9 100644
--- a/lib/spack/docs/build_systems/pythonpackage.rst
+++ b/lib/spack/docs/build_systems/pythonpackage.rst
@@ -724,10 +724,9 @@ extends vs. depends_on
This is very similar to the naming dilemma above, with a slight twist.
As mentioned in the :ref:`Packaging Guide <packaging_extensions>`,
-``extends`` and ``depends_on`` are very similar, but ``extends`` adds
-the ability to *activate* the package. Activation involves symlinking
-everything in the installation prefix of the package to the installation
-prefix of Python. This allows the user to import a Python module without
+``extends`` and ``depends_on`` are very similar, but ``extends`` ensures
+that the extension and extendee share the same prefix in views.
+This allows the user to import a Python module without
having to add that module to ``PYTHONPATH``.
When deciding between ``extends`` and ``depends_on``, the best rule of
@@ -735,7 +734,7 @@ thumb is to check the installation prefix. If Python libraries are
installed to ``<prefix>/lib/pythonX.Y/site-packages``, then you
should use ``extends``. If Python libraries are installed elsewhere
or the only files that get installed reside in ``<prefix>/bin``, then
-don't use ``extends``, as symlinking the package wouldn't be useful.
+don't use ``extends``.
^^^^^^^^^^^^^^^^^^^^^
Alternatives to Spack
diff --git a/lib/spack/docs/build_systems/rpackage.rst b/lib/spack/docs/build_systems/rpackage.rst
index ebf2270e8e..567b79a863 100644
--- a/lib/spack/docs/build_systems/rpackage.rst
+++ b/lib/spack/docs/build_systems/rpackage.rst
@@ -193,10 +193,10 @@ Build system dependencies
As an extension of the R ecosystem, your package will obviously depend
on R to build and run. Normally, we would use ``depends_on`` to express
-this, but for R packages, we use ``extends``. ``extends`` is similar to
-``depends_on``, but adds an additional feature: the ability to "activate"
-the package by symlinking it to the R installation directory. Since
-every R package needs this, the ``RPackage`` base class contains:
+this, but for R packages, we use ``extends``. This implies a special
+dependency on R, which is used to set environment variables such as
+``R_LIBS`` uniformly. Since every R package needs this, the ``RPackage``
+base class contains:
.. code-block:: python
diff --git a/lib/spack/docs/packaging_guide.rst b/lib/spack/docs/packaging_guide.rst
index fb1d5edcab..5b0e556b34 100644
--- a/lib/spack/docs/packaging_guide.rst
+++ b/lib/spack/docs/packaging_guide.rst
@@ -2634,9 +2634,12 @@ extendable package:
extends('python')
...
-Now, the ``py-numpy`` package can be used as an argument to ``spack
-activate``. When it is activated, all the files in its prefix will be
-symbolically linked into the prefix of the python package.
+This accomplishes a few things. Firstly, the Python package can set special
+variables such as ``PYTHONPATH`` for all extensions when the run or build
+environment is set up. Secondly, filesystem views can ensure that extensions
+are put in the same prefix as their extendee. This ensures that Python in
+a view can always locate its Python packages, even without environment
+variables set.
A package can only extend one other package at a time. To support packages
that may extend one of a list of other packages, Spack supports multiple
@@ -2684,9 +2687,8 @@ variant(s) are selected. This may be accomplished with conditional
...
Sometimes, certain files in one package will conflict with those in
-another, which means they cannot both be activated (symlinked) at the
-same time. In this case, you can tell Spack to ignore those files
-when it does the activation:
+another, which means they cannot both be used in a view at the
+same time. In this case, you can tell Spack to ignore those files:
.. code-block:: python
@@ -2698,7 +2700,7 @@ when it does the activation:
...
The code above will prevent everything in the ``$prefix/bin/`` directory
-from being linked in at activation time.
+from being linked in a view.
.. note::