docs: document releases and branches in Spack

- [x] Remove references to `master` branch
- [x] Document how release branches are structured
- [x] Document how to make a major release
- [x] Document how to make a point release
- [x] Document how to do work in our release projects

5 files changed, 384 insertions, 6 deletions

diff --git a/lib/spack/docs/contribution_guide.rst b/lib/spack/docs/contribution_guide.rst
index 9935ec0c83..10c0875e85 100644
--- a/lib/spack/docs/contribution_guide.rst
+++ b/lib/spack/docs/contribution_guide.rst
@@ -27,11 +27,22 @@ correspond to one feature/bugfix/extension/etc. One can create PRs with
 changes relevant to different ideas, however reviewing such PRs becomes tedious
 and error prone. If possible, try to follow the **one-PR-one-package/feature** rule.
 
-Spack uses a rough approximation of the `Git Flow <http://nvie.com/posts/a-successful-git-branching-model/>`_
-branching model. The develop branch contains the latest contributions, and
-master is always tagged and points to the latest stable release. Therefore, when
-you send your request, make ``develop`` the destination branch on the
-`Spack repository <https://github.com/spack/spack>`_.
+--------
+Branches
+--------
+
+Spack's ``develop`` branch has the latest contributions. Nearly all pull
+requests should start from ``develop`` and target ``develop``.
+
+There is a branch for each major release series. Release branches
+originate from ``develop`` and have tags for each point release in the
+series. For example, ``releases/v0.14`` has tags for ``0.14.0``,
+``0.14.1``, ``0.14.2``, etc. versions of Spack. We backport important bug
+fixes to these branches, but we do not advance the package versions or
+make other changes that would change the way Spack concretizes
+dependencies. Currently, the maintainers manage these branches by
+cherry-picking from ``develop``. See :ref:`releases` for more
+information.
 
 ----------------------
 Continuous Integration
diff --git a/lib/spack/docs/developer_guide.rst b/lib/spack/docs/developer_guide.rst
index de2fe80f85..5c57f28226 100644
--- a/lib/spack/docs/developer_guide.rst
+++ b/lib/spack/docs/developer_guide.rst
@@ -495,3 +495,370 @@ The bottom of the output shows the top most time consuming functions,
 slowest on top.  The profiling support is from Python's built-in tool,
 `cProfile
 <https://docs.python.org/2/library/profile.html#module-cProfile>`_.
+
+.. _releases:
+
+--------
+Releases
+--------
+
+This section documents Spack's release process. It is intended for
+project maintainers, as the tasks described here require maintainer
+privileges on the Spack repository. For others, we hope this section at
+least provides some insight into how the Spack project works.
+
+.. _release-branches:
+
+^^^^^^^^^^^^^^^^
+Release branches
+^^^^^^^^^^^^^^^^
+
+There are currently two types of Spack releases: :ref:`major releases
+<major-releases>` (``0.13.0``, ``0.14.0``, etc.) and :ref:`point releases
+<point-releases>` (``0.13.1``, ``0.13.2``, ``0.13.3``, etc.). Here is a
+diagram of how Spack release branches work::
+
+    o    branch: develop  (latest version)
+    |
+    o    merge v0.14.1 into develop
+    |\
+    | o  branch: releases/v0.14, tag: v0.14.1
+    o |  merge v0.14.0 into develop
+    |\|
+    | o  tag: v0.14.0
+    |/
+    o    merge v0.13.2 into develop
+    |\
+    | o  branch: releases/v0.13, tag: v0.13.2
+    o |  merge v0.13.1 into develop
+    |\|
+    | o  tag: v0.13.1
+    o |  merge v0.13.0 into develop
+    |\|
+    | o  tag: v0.13.0
+    o |
+    | o
+    |/
+    o
+
+The ``develop`` branch has the latest contributions, and nearly all pull
+requests target ``develop``.
+
+Each Spack release series also has a corresponding branch, e.g.
+``releases/v0.14`` has ``0.14.x`` versions of Spack, and
+``releases/v0.13`` has ``0.13.x`` versions. A major release is the first
+tagged version on a release branch. Minor releases are back-ported from
+develop onto release branches. This is typically done by cherry-picking
+bugfix commits off of ``develop``.
+
+To avoid version churn for users of a release series, minor releases
+should **not** make changes that would change the concretization of
+packages. They should generally only contain fixes to the Spack core.
+
+Both major and minor releases are tagged. After each release, we merge
+the release branch back into ``develop`` so that the version bump and any
+other release-specific changes are visible in the mainline (see
+:ref:`merging-releases-to-develop`).
+
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Scheduling work for releases
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+We schedule work for releases by creating `GitHub projects
+<https://github.com/spack/spack/projects>`_. At any time, there may be
+several open release projects. For example, here are two releases (from
+some past version of the page linked above):
+
+.. image:: images/projects.png
+
+Here, there's one release in progress for ``0.15.1`` and another for
+``0.16.0``. Each of these releases has a project board containing issues
+and pull requests. GitHub shows a status bar with completed work in
+green, work in progress in purple, and work not started yet in gray, so
+it's fairly easy to see progress.
+
+Spack's project boards are not firm commitments, and we move work between
+releases frequently. If we need to make a release and some tasks are not
+yet done, we will simply move them to next minor or major release, rather
+than delaying the release to complete them.
+
+For more on using GitHub project boards, see `GitHub's documentation
+<https://docs.github.com/en/github/managing-your-work-on-github/about-project-boards>`_.
+
+.. _major-releases:
+
+^^^^^^^^^^^^^^^^^^^^^
+Making Major Releases
+^^^^^^^^^^^^^^^^^^^^^
+
+Assuming you've already created a project board and completed the work
+for a major release, the steps to make the release are as follows:
+
+#. Create two new project boards:
+
+   * One for the next major release
+   * One for the next point release
+
+#. Move any tasks that aren't done yet to one of the new project boards.
+   Small bugfixes should go to the next point release. Major features,
+   refactors, and changes that could affect concretization should go in
+   the next major release.
+
+#. Create a branch for the release, based on ``develop``:
+
+   .. code-block:: console
+
+      $ git checkout -b releases/v0.15 develop
+
+   For a version ``vX.Y.Z``, the branch's name should be
+   ``releases/vX.Y``. That is, you should create a ``releases/vX.Y``
+   branch if you are preparing the ``X.Y.0`` release.
+
+#. Bump the version in ``lib/spack/spack/__init__.py``. See `this example from 0.13.0
+   <https://github.com/spack/spack/commit/8eeb64096c98b8a43d1c587f13ece743c864fba9>`_
+
+#. Updaate the release version lists in these files to include the new version:
+
+   * ``lib/spack/spack/schema/container.py``
+   * ``lib/spack/spack/container/images.json``
+
+   **TODO**: We should get rid of this step in some future release.
+
+#. Update ``CHANGELOG.md`` with major highlights in bullet form. Use
+   proper markdown formatting, like `this example from 0.15.0
+   <https://github.com/spack/spack/commit/d4bf70d9882fcfe88507e9cb444331d7dd7ba71c>`_.
+
+#. Push the release branch to GitHub.
+
+#. Make sure CI passes on the release branch, including:
+   * Regular unit tests
+   * Build tests
+   * The E4S pipeline at `gitlab.spack.io <https://gitlab.spack.io>`_
+
+   If CI is not passing, submit pull requests to ``develop`` as normal
+   and keep rebasing the release branch on ``develop`` until CI passes.
+
+#. Follow the steps in :ref:`publishing-releases`.
+
+#. Follow the steps in :ref:`merging-releases-to-develop`.
+
+#. Follow the steps in :ref:`announcing-releases`.
+
+
+.. _point-releases:
+
+^^^^^^^^^^^^^^^^^^^^^
+Making Point Releases
+^^^^^^^^^^^^^^^^^^^^^
+
+This assumes you've already created a project board for a point release
+and completed the work to be done for the release. To make a point
+release:
+
+#. Create one new project board for the next point release.
+
+#. Move any cards that aren't done yet to the next project board.
+
+#. Check out the release branch (it should already exist). For the
+   ``X.Y.Z`` release, the release branch is called ``releases/vX.Y``. For
+   ``v0.15.1``, you would check out ``releases/v0.15``:
+
+   .. code-block:: console
+
+      $ git checkout releases/v0.15
+
+#. Cherry-pick each pull request in the ``Done`` column of the release
+   project onto the release branch.
+
+   This is **usually** fairly simple since we squash the commits from the
+   vast majority of pull requests, which means there is only one commit
+   per pull request to cherry-pick. For example, `this pull request
+   <https://github.com/spack/spack/pull/15777>`_ has three commits, but
+   the were squashed into a single commit on merge. You can see the
+   commit that was created here:
+
+   .. image:: images/pr-commit.png
+
+   You can easily cherry pick it like this (assuming you already have the
+   release branch checked out):
+
+   .. code-block:: console
+
+      $ git cherry-pick 7e46da7
+
+   For pull requests that were rebased, you'll need to cherry-pick each
+   rebased commit individually. There have not been any rebased PRs like
+   this in recent point releases.
+
+   .. warning::
+
+      It is important to cherry-pick commits in the order they happened,
+      otherwise you can get conflicts while cherry-picking. When
+      cherry-picking onto a point release, look at the merge date,
+      **not** the number of the pull request or the date it was opened.
+
+      Sometimes you may **still** get merge conflicts even if you have
+      cherry-picked all the commits in order. This generally means there
+      is some other intervening pull request that the one you're trying
+      to pick depends on. In these cases, you'll need to make a judgment
+      call:
+
+      1. If the dependency is small, you might just cherry-pick it, too.
+         If you do this, add it to the release board.
+
+      2. If it is large, then you may decide that this fix is not worth
+         including in a point release, in which case you should remove it
+         from the release project.
+
+      3. You can always decide to manually back-port the fix to the release
+         branch if neither of the above options makes sense, but this can
+         require a lot of work. It's seldom the right choice.
+
+#. Bump the version in ``lib/spack/spack/__init__.py``. See `this example from 0.14.1
+   <https://github.com/spack/spack/commit/ff0abb9838121522321df2a054d18e54b566b44a>`_.
+
+#. Updaate the release version lists in these files to include the new version:
+
+   * ``lib/spack/spack/schema/container.py``
+   * ``lib/spack/spack/container/images.json``
+
+   **TODO**: We should get rid of this step in some future release.
+
+#. Update ``CHANGELOG.md`` with a list of bugfixes. This is typically just a
+   summary of the commits you cherry-picked onto the release branch. See
+   `the changelog from 0.14.1
+   <https://github.com/spack/spack/commit/ff0abb9838121522321df2a054d18e54b566b44a>`_.
+
+#. Push the release branch to GitHub.
+
+#. Make sure CI passes on the release branch, including:
+   * Regular unit tests
+   * Build tests
+   * The E4S pipeline at `gitlab.spack.io <https://gitlab.spack.io>`_
+
+   If CI does not pass, you'll need to figure out why, and make changes
+   to the release branch until it does. You can make more commits, modify
+   or remove cherry-picked commits, or cherry-pick **more** from
+   ``develop`` to make this happen.
+
+#. Follow the steps in :ref:`publishing-releases`.
+
+#. Follow the steps in :ref:`merging-releases-to-develop`.
+
+#. Follow the steps in :ref:`announcing-releases`.
+
+
+.. _publishing-releases:
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Publishing a release on GitHub
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+#. Go to `github.com/spack/spack/releases
+   <https://github.com/spack/spack/releases>`_ and click ``Draft a new
+   release``.  Set the following:
+
+   * ``Tag version`` should start with ``v`` and contain *all three*
+     parts of the version, .g. ``v0.15.1``. This is the name of the tag
+     that will be created.
+
+   * ``Target`` should be the ``releases/vX.Y`` branch (e.g., ``releases/v0.15``).
+
+   * ``Release title`` should be ``vX.Y.Z`` (To match the tag, e.g., ``v0.15.1``).
+
+   * For the text, paste the latest release markdown from your ``CHANGELOG.md``.
+
+   You can save the draft and keep coming back to this as you prepare the release.
+
+#. When you are done, click ``Publish release``.
+
+#. Immediately after publishing, go back to
+   `github.com/spack/spack/releases
+   <https://github.com/spack/spack/releases>`_ and download the
+   auto-generated ``.tar.gz`` file for the release. It's the ``Source
+   code (tar.gz)`` link.
+
+#. Click ``Edit`` on the release you just did and attach the downloaded
+   release tarball as a binary. This does two things:
+
+   #. Makes sure that the hash of our releases doesn't change over time.
+      GitHub sometimes annoyingly changes they way they generate
+      tarballs, and then hashes can change if you rely on the
+      auto-generated tarball links.
+
+   #. Gets us download counts on releases visible through the GitHub
+      API. GitHub tracks downloads of artifacts, but *not* the source
+      links. See the `releases
+      page <https://api.github.com/repos/spack/spack/releases>`_ and search
+      for ``download_count`` to see this.
+
+
+.. _merging-releases-to-develop:
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Merging back into ``develop``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Once each release is complete, make sure that it is merged back into
+``develop`` with a merge commit:
+
+.. code-block:: console
+
+   $ git checkout develop
+   $ git merge --no-ff releases/v0.15
+   $ git push
+
+We merge back to ``develop`` because it:
+
+  * updates the version and ``CHANGELOG.md`` on ``develop``.
+  * ensures that your release tag is reachable from the head of
+    ``develop``
+
+We *must* use a real merge commit (via the ``--no-ff`` option) because it
+ensures that the release tag is reachable from the tip of ``develop``.
+This is necessary for ``spack -V`` to work properly -- it uses ``git
+describe --tags`` to find the last reachable tag in the repository and
+reports how far we are from it. For example:
+
+.. code-block:: console
+
+   $ spack -V
+   0.14.2-1486-b80d5e74e5
+
+This says that we are at commit ``b80d5e74e5``, which is 1,486 commits
+ahead of the ``0.14.2`` release.
+
+We put this step last in the process because it's best to do it only once
+the release is complete and tagged. If you do it before you've tagged the
+release and later decide you want to tag some later commit, you'll need
+to merge again.
+
+
+.. _announcing-releases:
+
+^^^^^^^^^^^^^^^^^^^^
+Announcing a release
+^^^^^^^^^^^^^^^^^^^^
+
+We announce releases in all of the major Spack communication channels.
+Publishing the release takes care of GitHub. The remaining channels are
+Twitter, Slack, and the mailing list. Here are the steps:
+
+#. Make a tweet to announce the release. It should link to the release's
+   page on GitHub. You can base it on `this example tweet
+   <https://twitter.com/spackpm/status/1231761858182307840>`_.
+
+#. Ping ``@channel`` in ``#general`` on Slack (`spackpm.slack.com
+   <https://spackpm.slack.com>`_) with a link to the tweet. The tweet
+   will be shown inline so that you do not have to retype your release
+   announcement.
+
+#. Email the Spack mailing list to let them know about the release. As
+   with the tweet, you likely want to link to the release's page on
+   GitHub. It's also helpful to include some information directly in the
+   email. You can base yours on this `example email
+   <https://groups.google.com/forum/#!topic/spack/WT4CT9i_X4s>`_.
+
+Once you've announced the release, congratulations, you're done! You've
+finished making the release!
diff --git a/lib/spack/docs/getting_started.rst b/lib/spack/docs/getting_started.rst
index 226b1f0883..7b908465f5 100644
--- a/lib/spack/docs/getting_started.rst
+++ b/lib/spack/docs/getting_started.rst
@@ -818,7 +818,7 @@ Git
 
 Some Spack packages use ``git`` to download, which might not work on
 some computers.  For example, the following error was
-encountered on a Macintosh during ``spack install julia-master``:
+encountered on a Macintosh during ``spack install julia@master``:
 
 .. code-block:: console
 
diff --git a/lib/spack/docs/images/pr-commit.png b/lib/spack/docs/images/pr-commit.png
new file mode 100644
index 0000000000..a87c800ef5
--- /dev/null
+++ b/lib/spack/docs/images/pr-commit.png
diff --git a/lib/spack/docs/images/projects.png b/lib/spack/docs/images/projects.png
new file mode 100644
index 0000000000..bd2971ff62
--- /dev/null
+++ b/lib/spack/docs/images/projects.png