Documentation: Refinement of "Checking an installation" (#22210)

There have been a lot of questions and some confusion recently surrounding Spack installation test capabilities so this PR is intended to clean up and refine the documentation for "Checking an installation".

It aims to better distinguish between checks that are performed during an installation (i.e., build-time tests) and those that can be done days and weeks after the software has been installed (i.e., install (or smoke) tests).

2 files changed, 31 insertions, 16 deletions

diff --git a/lib/spack/docs/build_systems/custompackage.rst b/lib/spack/docs/build_systems/custompackage.rst
index b513e3963a..4cb506acd7 100644
--- a/lib/spack/docs/build_systems/custompackage.rst
+++ b/lib/spack/docs/build_systems/custompackage.rst
@@ -9,7 +9,7 @@
 Custom Build Systems
 --------------------
 
-While the build systems listed above should meet your needs for the
+While the built-in build systems should meet your needs for the
 vast majority of packages, some packages provide custom build scripts.
 This guide is intended for the following use cases:
 
@@ -31,7 +31,7 @@ installation. Both of these packages require custom build systems.
 Base class
 ^^^^^^^^^^
 
-If your package does not belong to any of the aforementioned build
+If your package does not belong to any of the built-in build
 systems that Spack already supports, you should inherit from the
 ``Package`` base class. ``Package`` is a simple base class with a
 single phase: ``install``. If your package is simple, you may be able
@@ -168,7 +168,8 @@ if and only if this flag is set, we would use the following line:
 Testing
 ^^^^^^^
 
-Let's put everything together and add unit tests to our package.
+Let's put everything together and add unit tests to be optionally run
+during the installation of our package.
 In the ``perl`` package, we can see:
 
 .. code-block:: python
@@ -182,12 +183,6 @@ As you can guess, this runs ``make test`` *after* building the package,
 if and only if testing is requested. Again, this is not specific to
 custom build systems, it can be added to existing build systems as well.
 
-Ideally, every package in Spack will have some sort of test to ensure
-that it was built correctly. It is up to the package authors to make
-sure this happens. If you are adding a package for some software and
-the developers list commands to test the installation, please add these
-tests to your ``package.py``.
-
 .. warning::
 
    The order of decorators matters. The following ordering:
@@ -207,3 +202,12 @@ tests to your ``package.py``.
    the tests will always be run regardless of whether or not
    ``--test=root`` is requested. See https://github.com/spack/spack/issues/3833
    for more information
+
+Ideally, every package in Spack will have some sort of test to ensure
+that it was built correctly. It is up to the package authors to make
+sure this happens. If you are adding a package for some software and
+the developers list commands to test the installation, please add these
+tests to your ``package.py``.
+
+For more information on other forms of package testing, refer to
+:ref:`Checking an installation <checking_an_installation>`.
diff --git a/lib/spack/docs/build_systems/pythonpackage.rst b/lib/spack/docs/build_systems/pythonpackage.rst
index f1c85d92cb..931c58724a 100644
--- a/lib/spack/docs/build_systems/pythonpackage.rst
+++ b/lib/spack/docs/build_systems/pythonpackage.rst
@@ -627,7 +627,8 @@ adds:
 Testing
 ^^^^^^^
 
-``PythonPackage`` provides a couple of options for testing packages.
+``PythonPackage`` provides a couple of options for testing packages
+both during and after the installation process.
 
 """"""""""""
 Import tests
@@ -696,16 +697,20 @@ libraries. Make sure not to add modules/packages containing the word
 "test", as these likely won't end up in the installation directory,
 or may require test dependencies like pytest to be installed.
 
-These tests can be triggered by running ``spack install --test=root``
-or by running ``spack test run`` after the installation has finished.
+Import tests can be run during the installation using ``spack install
+--test=root`` or at any time after the installation using
+``spack test run``.
 
 """"""""""
 Unit tests
 """"""""""
 
-The package you want to install may come with additional unit tests.
-You can add additional build-time or install-time tests by adding
-additional testing functions. For example, ``py-numpy`` adds:
+The package may have its own unit or regression tests. Spack can
+run these tests during the installation by adding phase-appropriate
+test methods.
+
+For example, ``py-numpy`` adds the following as a check to run 
+after the ``install`` phase:
 
 .. code-block:: python
 
@@ -716,7 +721,13 @@ additional testing functions. For example, ``py-numpy`` adds:
            python('-c', 'import numpy; numpy.test("full", verbose=2)')
 
 
-These tests can be triggered by running ``spack install --test=root``.
+when testing is enabled during the installation (i.e., ``spack install
+--test=root``).
+
+.. note::
+
+   Additional information is available on :ref:`install phase tests
+   <install_phase-tests>`.
 
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Setup file in a sub-directory