Update docs on "spack external find" (#16482)

This improves the documentation for `spack external find` in several ways:

* Provide a code example of implementing `determine_spec_details` for a package
* Explain how to define executables to look for (and also e.g. that they are treated as regular expressions and so can pull in unexpected files).
* Add the "why" for a couple of constraints (i.e. explain that this logic only works for build/run deps because it examines `PATH` for executables)
* Spread the docs between build customization and packaging sections
* Add cross-references
* Add a label so that `spack external find` is linked from the command reference.

2 files changed, 80 insertions, 8 deletions

diff --git a/lib/spack/docs/build_settings.rst b/lib/spack/docs/build_settings.rst
index dee892f272..9f67d8c14f 100644
--- a/lib/spack/docs/build_settings.rst
+++ b/lib/spack/docs/build_settings.rst
@@ -158,11 +158,13 @@ Spack can then use any of the listed external implementations of MPI
 to satisfy a dependency, and will choose depending on the compiler and
 architecture.
 
+.. _cmd-spack-external-find:
+
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Automatically Find External Packages
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-A user can run the :ref:`spack external find <spack-external-find>` command
+You can run the :ref:`spack external find <spack-external-find>` command
 to search for system-provided packages and add them to ``packages.yaml``.
 After running this command your ``packages.yaml`` may include new entries:
 
@@ -177,17 +179,23 @@ Generally this is useful for detecting a small set of commonly-used packages;
 for now this is generally limited to finding build-only dependencies.
 Specific limitations include:
 
-* A package must define ``executables`` and ``determine_spec_details``
-  for Spack to locate instances of that package.
-* This is currently intended to find build dependencies rather than
-  library packages.
+* Packages are not discoverable by default: For a package to be
+  discoverable with ``spack external find``, it needs to add special
+  logic. See :ref:`here <make-package-findable>` for more details.
+* The current implementation only collects and examines executable files,
+  so it is typically only useful for build/run dependencies (in some cases
+  if a library package also provides an executable, it may be possible to
+  extract a meaningful Spec by running the executable - for example the
+  compiler wrappers in MPI implementations).
+* The logic does not search through module files, it can only detect
+  packages with executables defined in ``PATH``; you can help Spack locate
+  externals which use module files by loading any associated modules for
+  packages that you want Spack to know about before running
+  ``spack external find``.
 * Spack does not overwrite existing entries in the package configuration:
   If there is an external defined for a spec at any configuration scope,
   then Spack will not add a new external entry (``spack config blame packages``
   can help locate all external entries).
-* Currently this logic is focused on examining ``PATH`` and does not
-  search through modules (although it should find the package if a
-  module is loaded for it).
 
 .. _concretization-preferences:
 
diff --git a/lib/spack/docs/packaging_guide.rst b/lib/spack/docs/packaging_guide.rst
index 7abadaac82..840c29454b 100644
--- a/lib/spack/docs/packaging_guide.rst
+++ b/lib/spack/docs/packaging_guide.rst
@@ -4048,6 +4048,70 @@ File functions
 :py:func:`touch(path) <spack.touch>`
   Create an empty file at ``path``.
 
+.. _make-package-findable:
+
+----------------------------------------------------------
+Making a package discoverable with ``spack external find``
+----------------------------------------------------------
+
+To make a package discoverable with
+:ref:`spack external find <cmd-spack-external-find>` you must
+define one or more executables associated with the package and must
+implement a method to generate a Spec when given an executable.
+
+The executables are specified as a package level ``executables``
+attribute which is a list of strings (see example below); each string
+is treated as a regular expression (e.g. 'gcc' would match 'gcc', 'gcc-8.3',
+'my-weird-gcc', etc.).
+
+The method ``determine_spec_details`` has the following signature:
+
+.. code-block:: python
+
+   def determine_spec_details(prefix, exes_in_prefix):
+       # exes_in_prefix = a set of paths, each path is an executable
+       # prefix = a prefix that is common to each path in exes_in_prefix
+
+       # return None or [] if none of the exes represent an instance of
+       # the package. Return one or more Specs for each instance of the
+       # package which is thought to be installed in the provided prefix
+
+``determine_spec_details`` takes as parameters a set of discovered
+executables (which match those specified by the user) as well as a
+common prefix shared by all of those executables. The function must
+return one or more Specs associated with the executables (it can also
+return ``None`` to indicate that no provided executables are associated
+with the package).
+
+Say for example we have a package called ``foo-package`` which
+builds an executable called ``foo``. ``FooPackage`` would appear as
+follows:
+
+.. code-block:: python
+
+   class FooPackage(Package):
+       homepage = "..."
+       url = "..."
+
+       version(...)
+
+       # Each string provided here is treated as a regular expression, and
+       # would match for example 'foo', 'foobar', and 'bazfoo'.
+       executables = ['foo']
+
+       @classmethod
+       def determine_spec_details(cls, prefix, exes_in_prefix):
+           candidates = list(x for x in exes_in_prefix
+                             if os.path.basename(x) == 'foo')
+           if not candidates:
+               return
+           # This implementation is lazy and only checks the first candidate
+           exe_path = candidates[0]
+           exe = spack.util.executable.Executable(exe_path)
+           output = exe('--version')
+           version_str = ...  # parse output for version string
+           return Spec('foo-package@{0}'.format(version_str))
+
 .. _package-lifecycle:
 
 -----------------------------