Add documentation for git refs as versions (#31914)

* document git commit versions

Include documentation for manually specifying associated known version

* document spack develop command

Co-authorerd-by: psakievich <psakiev@sandia.gov>
Co-authored-by: Gregory Becker <becker33@llnl.gov>
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>

3 files changed, 133 insertions, 1 deletions

diff --git a/lib/spack/docs/basic_usage.rst b/lib/spack/docs/basic_usage.rst
index 7a3c44a759..184d7feca6 100644
--- a/lib/spack/docs/basic_usage.rst
+++ b/lib/spack/docs/basic_usage.rst
@@ -1093,6 +1093,8 @@ could depend on ``mpich@1.2:`` if it can only build with version
 
 Below are more details about the specifiers that you can add to specs.
 
+.. _version-specifier:
+
 ^^^^^^^^^^^^^^^^^
 Version specifier
 ^^^^^^^^^^^^^^^^^
@@ -1108,6 +1110,37 @@ set of arbitrary versions, such as ``@1.0,1.5,1.7`` (``1.0``, ``1.5``,
 or ``1.7``).  When you supply such a specifier to ``spack install``,
 it constrains the set of versions that Spack will install.
 
+For packages with a ``git`` attribute, ``git`` references 
+may be specified instead of a numerical version i.e. branches, tags 
+and commits. Spack will stage and build based off the ``git`` 
+reference provided.  Acceptable syntaxes for this are:
+
+.. code-block:: console
+   
+    # branches and tags
+   foo@git.develop # use the develop branch
+   foo@git.0.19 # use the 0.19 tag
+   
+    # commit hashes
+   foo@abcdef1234abcdef1234abcdef1234abcdef1234    # 40 character hashes are automatically treated as git commits
+   foo@git.abcdef1234abcdef1234abcdef1234abcdef1234
+   
+Spack versions from git reference either have an associated version supplied by the user,
+or infer a relationship to known versions from the structure of the git repository. If an
+associated version is supplied by the user, Spack treats the git version as equivalent to that
+version for all version comparisons in the package logic (e.g. ``depends_on('foo', when='@1.5')``).
+
+The associated version can be assigned with ``[git ref]=[version]`` syntax, with the caveat that the specified version is known to Spack from either the package definition, or in the configuration preferences (i.e. ``packages.yaml``).
+
+.. code-block:: console
+
+   foo@git.my_ref=3.2 # use the my_ref tag or branch, but treat it as version 3.2 for version comparisons
+   foo@git.abcdef1234abcdef1234abcdef1234abcdef1234=develop # use the given commit, but treat it as develop for version comparisons
+
+If an associated version is not supplied then the tags in the git repo are used to determine
+the most recent previous version known to Spack. Details about how versions are compared
+and how Spack determines if one version is less than another are discussed in the developer guide.
+
 If the version spec is not provided, then Spack will choose one
 according to policies set for the particular spack installation.  If
 the spec is ambiguous, i.e. it could match multiple versions, Spack
diff --git a/lib/spack/docs/environments.rst b/lib/spack/docs/environments.rst
index ee95d4f779..4c3f6d1971 100644
--- a/lib/spack/docs/environments.rst
+++ b/lib/spack/docs/environments.rst
@@ -376,6 +376,30 @@ from being added again.  At the same time, a spec that already exists in the
 environment, but only as a dependency, will be added to the environment as a
 root spec without the ``--no-add`` option.
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Developing Packages in a Spack Environment
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``spack develop`` command allows one to develop Spack packages in
+an environment. It requires a spec containing a concrete version, and
+will configure Spack to install the package from local source. By
+default, it will also clone the package to a subdirectory in the
+environment. This package will have a special variant ``dev_path``
+set, and Spack will ensure the package and its dependents are rebuilt
+any time the environment is installed if the package's local source
+code has been modified. Spack ensures that all instances of a
+developed package in the environment are concretized to match the
+version (and other constraints) passed as the spec argument to the
+``spack develop`` command.
+
+For packages with ``git`` attributes, git branches, tags, and commits can
+also be used as valid concrete versions (see :ref:`version-specifier`).
+This means that for a package ``foo``, ``spack develop foo@git.main`` will clone 
+the ``main`` branch of the package, and ``spack install`` will install from
+that git clone if ``foo`` is in the environment.
+Further development on ``foo`` can be tested by reinstalling the environment,
+and eventually committed and pushed to the upstream git repo.
+
 ^^^^^^^
 Loading
 ^^^^^^^
diff --git a/lib/spack/spack/version.py b/lib/spack/spack/version.py
index 89c7414abd..e67ab260ba 100644
--- a/lib/spack/spack/version.py
+++ b/lib/spack/spack/version.py
@@ -193,7 +193,43 @@ def Version(string):  # capitalized for backwards compatibility
 
 
 class VersionBase(object):
-    """Class to represent versions"""
+    """Class to represent versions
+
+    Versions are compared by converting to a tuple and comparing
+    lexicographically.
+
+    The most common Versions are alpha-numeric, and are parsed from strings
+    such as ``2.3.0`` or ``1.2-5``. These Versions are represented by
+    the tuples ``(2, 3, 0)`` and ``(1, 2, 5)`` respectively.
+
+    Versions are split on ``.``, ``-``, and
+    ``_`` characters, as well as any point at which they switch from
+    numeric to alphabetical or vice-versa. For example, the version
+    ``0.1.3a`` is represented by the tuple ``(0, 1, 3, 'a') and the
+    version ``a-5b`` is represented by the tuple ``('a', 5, 'b')``.
+
+    Spack versions may also be arbitrary non-numeric strings or git
+    commit SHAs. The following are the full rules for comparing
+    versions.
+
+    1. If the version represents a git reference (i.e. commit, tag, branch), see GitVersions.
+
+    2. The version is split into fields based on the delimiters ``.``,
+    ``-``, and ``_``, as well as alphabetic-numeric boundaries.
+
+    3. The following develop-like strings are greater (newer) than all
+    numbers and are ordered as ``develop > main > master > head >
+    trunk``.
+
+    4. All other non-numeric versions are less than numeric versions,
+    and are sorted alphabetically.
+
+    These rules can be summarized as follows:
+
+    ``develop > main > master > head > trunk > 999 > 0 > 'zzz' > 'a' >
+    ''``
+
+    """
 
     __slots__ = [
         "version",
@@ -463,6 +499,45 @@ class VersionBase(object):
 class GitVersion(VersionBase):
     """Class to represent versions interpreted from git refs.
 
+    There are two distinct categories of git versions:
+
+    1) GitVersions instantiated with an associated reference version (e.g. 'git.foo=1.2')
+    2) GitVersions requiring commit lookups
+
+    Git ref versions that are not paried with a known version
+    are handled separately from all other version comparisons.
+    When Spack identifies a git ref version, it associates a
+    ``CommitLookup`` object with the version. This object
+    handles caching of information from the git repo. When executing
+    comparisons with a git ref version, Spack queries the
+    ``CommitLookup`` for the most recent version previous to this
+    git ref, as well as the distance between them expressed as a number
+    of commits. If the previous version is ``X.Y.Z`` and the distance
+    is ``D``, the git commit version is represented by the tuple ``(X,
+    Y, Z, '', D)``. The component ``''`` cannot be parsed as part of
+    any valid version, but is a valid component. This allows a git
+    ref version to be less than (older than) every Version newer
+    than its previous version, but still newer than its previous
+    version.
+
+    To find the previous version from a git ref version, Spack
+    queries the git repo for its tags. Any tag that matches a version
+    known to Spack is associated with that version, as is any tag that
+    is a known version prepended with the character ``v`` (i.e., a tag
+    ``v1.0`` is associated with the known version
+    ``1.0``). Additionally, any tag that represents a semver version
+    (X.Y.Z with X, Y, Z all integers) is associated with the version
+    it represents, even if that version is not known to Spack. Each
+    tag is then queried in git to see whether it is an ancestor of the
+    git ref in question, and if so the distance between the two. The
+    previous version is the version that is an ancestor with the least
+    distance from the git ref in question.
+
+    This procedure can be circumvented if the user supplies a known version
+    to associate with the GitVersion (e.g. ``[hash]=develop``).  If the user
+    prescribes the version then there is no need to do a lookup
+    and the standard version comparison operations are sufficient.
+
     Non-git versions may be coerced to GitVersion for comparison, but no Spec will ever
     have a GitVersion that is not actually referencing a version from git."""