summaryrefslogtreecommitdiff
path: root/lib
diff options
context:
space:
mode:
authorTodd Gamblin <tgamblin@llnl.gov>2020-07-04 01:50:55 -0700
committerTodd Gamblin <tgamblin@llnl.gov>2020-08-13 00:33:31 -0700
commit303882834ac1ace64fcad9cebf7d0ffc228d76a9 (patch)
treea0801ed28d0118b3a860512372a31f4e29dbb3ba /lib
parent5b63ec86528d52d7bfa86bef9abdf263d2ddc9bf (diff)
downloadspack-303882834ac1ace64fcad9cebf7d0ffc228d76a9.tar.gz
spack-303882834ac1ace64fcad9cebf7d0ffc228d76a9.tar.bz2
spack-303882834ac1ace64fcad9cebf7d0ffc228d76a9.tar.xz
spack-303882834ac1ace64fcad9cebf7d0ffc228d76a9.zip
docs: document releases and branches in Spack
- [x] Remove references to `master` branch - [x] Document how release branches are structured - [x] Document how to make a major release - [x] Document how to make a point release - [x] Document how to do work in our release projects
Diffstat (limited to 'lib')
-rw-r--r--lib/spack/docs/contribution_guide.rst21
-rw-r--r--lib/spack/docs/developer_guide.rst367
-rw-r--r--lib/spack/docs/getting_started.rst2
-rw-r--r--lib/spack/docs/images/pr-commit.pngbin0 -> 44592 bytes
-rw-r--r--lib/spack/docs/images/projects.pngbin0 -> 69075 bytes
5 files changed, 384 insertions, 6 deletions
diff --git a/lib/spack/docs/contribution_guide.rst b/lib/spack/docs/contribution_guide.rst
index 9935ec0c83..10c0875e85 100644
--- a/lib/spack/docs/contribution_guide.rst
+++ b/lib/spack/docs/contribution_guide.rst
@@ -27,11 +27,22 @@ correspond to one feature/bugfix/extension/etc. One can create PRs with
changes relevant to different ideas, however reviewing such PRs becomes tedious
and error prone. If possible, try to follow the **one-PR-one-package/feature** rule.
-Spack uses a rough approximation of the `Git Flow <http://nvie.com/posts/a-successful-git-branching-model/>`_
-branching model. The develop branch contains the latest contributions, and
-master is always tagged and points to the latest stable release. Therefore, when
-you send your request, make ``develop`` the destination branch on the
-`Spack repository <https://github.com/spack/spack>`_.
+--------
+Branches
+--------
+
+Spack's ``develop`` branch has the latest contributions. Nearly all pull
+requests should start from ``develop`` and target ``develop``.
+
+There is a branch for each major release series. Release branches
+originate from ``develop`` and have tags for each point release in the
+series. For example, ``releases/v0.14`` has tags for ``0.14.0``,
+``0.14.1``, ``0.14.2``, etc. versions of Spack. We backport important bug
+fixes to these branches, but we do not advance the package versions or
+make other changes that would change the way Spack concretizes
+dependencies. Currently, the maintainers manage these branches by
+cherry-picking from ``develop``. See :ref:`releases` for more
+information.
----------------------
Continuous Integration
diff --git a/lib/spack/docs/developer_guide.rst b/lib/spack/docs/developer_guide.rst
index de2fe80f85..5c57f28226 100644
--- a/lib/spack/docs/developer_guide.rst
+++ b/lib/spack/docs/developer_guide.rst
@@ -495,3 +495,370 @@ The bottom of the output shows the top most time consuming functions,
slowest on top. The profiling support is from Python's built-in tool,
`cProfile
<https://docs.python.org/2/library/profile.html#module-cProfile>`_.
+
+.. _releases:
+
+--------
+Releases
+--------
+
+This section documents Spack's release process. It is intended for
+project maintainers, as the tasks described here require maintainer
+privileges on the Spack repository. For others, we hope this section at
+least provides some insight into how the Spack project works.
+
+.. _release-branches:
+
+^^^^^^^^^^^^^^^^
+Release branches
+^^^^^^^^^^^^^^^^
+
+There are currently two types of Spack releases: :ref:`major releases
+<major-releases>` (``0.13.0``, ``0.14.0``, etc.) and :ref:`point releases
+<point-releases>` (``0.13.1``, ``0.13.2``, ``0.13.3``, etc.). Here is a
+diagram of how Spack release branches work::
+
+ o branch: develop (latest version)
+ |
+ o merge v0.14.1 into develop
+ |\
+ | o branch: releases/v0.14, tag: v0.14.1
+ o | merge v0.14.0 into develop
+ |\|
+ | o tag: v0.14.0
+ |/
+ o merge v0.13.2 into develop
+ |\
+ | o branch: releases/v0.13, tag: v0.13.2
+ o | merge v0.13.1 into develop
+ |\|
+ | o tag: v0.13.1
+ o | merge v0.13.0 into develop
+ |\|
+ | o tag: v0.13.0
+ o |
+ | o
+ |/
+ o
+
+The ``develop`` branch has the latest contributions, and nearly all pull
+requests target ``develop``.
+
+Each Spack release series also has a corresponding branch, e.g.
+``releases/v0.14`` has ``0.14.x`` versions of Spack, and
+``releases/v0.13`` has ``0.13.x`` versions. A major release is the first
+tagged version on a release branch. Minor releases are back-ported from
+develop onto release branches. This is typically done by cherry-picking
+bugfix commits off of ``develop``.
+
+To avoid version churn for users of a release series, minor releases
+should **not** make changes that would change the concretization of
+packages. They should generally only contain fixes to the Spack core.
+
+Both major and minor releases are tagged. After each release, we merge
+the release branch back into ``develop`` so that the version bump and any
+other release-specific changes are visible in the mainline (see
+:ref:`merging-releases-to-develop`).
+
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Scheduling work for releases
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+We schedule work for releases by creating `GitHub projects
+<https://github.com/spack/spack/projects>`_. At any time, there may be
+several open release projects. For example, here are two releases (from
+some past version of the page linked above):
+
+.. image:: images/projects.png
+
+Here, there's one release in progress for ``0.15.1`` and another for
+``0.16.0``. Each of these releases has a project board containing issues
+and pull requests. GitHub shows a status bar with completed work in
+green, work in progress in purple, and work not started yet in gray, so
+it's fairly easy to see progress.
+
+Spack's project boards are not firm commitments, and we move work between
+releases frequently. If we need to make a release and some tasks are not
+yet done, we will simply move them to next minor or major release, rather
+than delaying the release to complete them.
+
+For more on using GitHub project boards, see `GitHub's documentation
+<https://docs.github.com/en/github/managing-your-work-on-github/about-project-boards>`_.
+
+.. _major-releases:
+
+^^^^^^^^^^^^^^^^^^^^^
+Making Major Releases
+^^^^^^^^^^^^^^^^^^^^^
+
+Assuming you've already created a project board and completed the work
+for a major release, the steps to make the release are as follows:
+
+#. Create two new project boards:
+
+ * One for the next major release
+ * One for the next point release
+
+#. Move any tasks that aren't done yet to one of the new project boards.
+ Small bugfixes should go to the next point release. Major features,
+ refactors, and changes that could affect concretization should go in
+ the next major release.
+
+#. Create a branch for the release, based on ``develop``:
+
+ .. code-block:: console
+
+ $ git checkout -b releases/v0.15 develop
+
+ For a version ``vX.Y.Z``, the branch's name should be
+ ``releases/vX.Y``. That is, you should create a ``releases/vX.Y``
+ branch if you are preparing the ``X.Y.0`` release.
+
+#. Bump the version in ``lib/spack/spack/__init__.py``. See `this example from 0.13.0
+ <https://github.com/spack/spack/commit/8eeb64096c98b8a43d1c587f13ece743c864fba9>`_
+
+#. Updaate the release version lists in these files to include the new version:
+
+ * ``lib/spack/spack/schema/container.py``
+ * ``lib/spack/spack/container/images.json``
+
+ **TODO**: We should get rid of this step in some future release.
+
+#. Update ``CHANGELOG.md`` with major highlights in bullet form. Use
+ proper markdown formatting, like `this example from 0.15.0
+ <https://github.com/spack/spack/commit/d4bf70d9882fcfe88507e9cb444331d7dd7ba71c>`_.
+
+#. Push the release branch to GitHub.
+
+#. Make sure CI passes on the release branch, including:
+ * Regular unit tests
+ * Build tests
+ * The E4S pipeline at `gitlab.spack.io <https://gitlab.spack.io>`_
+
+ If CI is not passing, submit pull requests to ``develop`` as normal
+ and keep rebasing the release branch on ``develop`` until CI passes.
+
+#. Follow the steps in :ref:`publishing-releases`.
+
+#. Follow the steps in :ref:`merging-releases-to-develop`.
+
+#. Follow the steps in :ref:`announcing-releases`.
+
+
+.. _point-releases:
+
+^^^^^^^^^^^^^^^^^^^^^
+Making Point Releases
+^^^^^^^^^^^^^^^^^^^^^
+
+This assumes you've already created a project board for a point release
+and completed the work to be done for the release. To make a point
+release:
+
+#. Create one new project board for the next point release.
+
+#. Move any cards that aren't done yet to the next project board.
+
+#. Check out the release branch (it should already exist). For the
+ ``X.Y.Z`` release, the release branch is called ``releases/vX.Y``. For
+ ``v0.15.1``, you would check out ``releases/v0.15``:
+
+ .. code-block:: console
+
+ $ git checkout releases/v0.15
+
+#. Cherry-pick each pull request in the ``Done`` column of the release
+ project onto the release branch.
+
+ This is **usually** fairly simple since we squash the commits from the
+ vast majority of pull requests, which means there is only one commit
+ per pull request to cherry-pick. For example, `this pull request
+ <https://github.com/spack/spack/pull/15777>`_ has three commits, but
+ the were squashed into a single commit on merge. You can see the
+ commit that was created here:
+
+ .. image:: images/pr-commit.png
+
+ You can easily cherry pick it like this (assuming you already have the
+ release branch checked out):
+
+ .. code-block:: console
+
+ $ git cherry-pick 7e46da7
+
+ For pull requests that were rebased, you'll need to cherry-pick each
+ rebased commit individually. There have not been any rebased PRs like
+ this in recent point releases.
+
+ .. warning::
+
+ It is important to cherry-pick commits in the order they happened,
+ otherwise you can get conflicts while cherry-picking. When
+ cherry-picking onto a point release, look at the merge date,
+ **not** the number of the pull request or the date it was opened.
+
+ Sometimes you may **still** get merge conflicts even if you have
+ cherry-picked all the commits in order. This generally means there
+ is some other intervening pull request that the one you're trying
+ to pick depends on. In these cases, you'll need to make a judgment
+ call:
+
+ 1. If the dependency is small, you might just cherry-pick it, too.
+ If you do this, add it to the release board.
+
+ 2. If it is large, then you may decide that this fix is not worth
+ including in a point release, in which case you should remove it
+ from the release project.
+
+ 3. You can always decide to manually back-port the fix to the release
+ branch if neither of the above options makes sense, but this can
+ require a lot of work. It's seldom the right choice.
+
+#. Bump the version in ``lib/spack/spack/__init__.py``. See `this example from 0.14.1
+ <https://github.com/spack/spack/commit/ff0abb9838121522321df2a054d18e54b566b44a>`_.
+
+#. Updaate the release version lists in these files to include the new version:
+
+ * ``lib/spack/spack/schema/container.py``
+ * ``lib/spack/spack/container/images.json``
+
+ **TODO**: We should get rid of this step in some future release.
+
+#. Update ``CHANGELOG.md`` with a list of bugfixes. This is typically just a
+ summary of the commits you cherry-picked onto the release branch. See
+ `the changelog from 0.14.1
+ <https://github.com/spack/spack/commit/ff0abb9838121522321df2a054d18e54b566b44a>`_.
+
+#. Push the release branch to GitHub.
+
+#. Make sure CI passes on the release branch, including:
+ * Regular unit tests
+ * Build tests
+ * The E4S pipeline at `gitlab.spack.io <https://gitlab.spack.io>`_
+
+ If CI does not pass, you'll need to figure out why, and make changes
+ to the release branch until it does. You can make more commits, modify
+ or remove cherry-picked commits, or cherry-pick **more** from
+ ``develop`` to make this happen.
+
+#. Follow the steps in :ref:`publishing-releases`.
+
+#. Follow the steps in :ref:`merging-releases-to-develop`.
+
+#. Follow the steps in :ref:`announcing-releases`.
+
+
+.. _publishing-releases:
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Publishing a release on GitHub
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+#. Go to `github.com/spack/spack/releases
+ <https://github.com/spack/spack/releases>`_ and click ``Draft a new
+ release``. Set the following:
+
+ * ``Tag version`` should start with ``v`` and contain *all three*
+ parts of the version, .g. ``v0.15.1``. This is the name of the tag
+ that will be created.
+
+ * ``Target`` should be the ``releases/vX.Y`` branch (e.g., ``releases/v0.15``).
+
+ * ``Release title`` should be ``vX.Y.Z`` (To match the tag, e.g., ``v0.15.1``).
+
+ * For the text, paste the latest release markdown from your ``CHANGELOG.md``.
+
+ You can save the draft and keep coming back to this as you prepare the release.
+
+#. When you are done, click ``Publish release``.
+
+#. Immediately after publishing, go back to
+ `github.com/spack/spack/releases
+ <https://github.com/spack/spack/releases>`_ and download the
+ auto-generated ``.tar.gz`` file for the release. It's the ``Source
+ code (tar.gz)`` link.
+
+#. Click ``Edit`` on the release you just did and attach the downloaded
+ release tarball as a binary. This does two things:
+
+ #. Makes sure that the hash of our releases doesn't change over time.
+ GitHub sometimes annoyingly changes they way they generate
+ tarballs, and then hashes can change if you rely on the
+ auto-generated tarball links.
+
+ #. Gets us download counts on releases visible through the GitHub
+ API. GitHub tracks downloads of artifacts, but *not* the source
+ links. See the `releases
+ page <https://api.github.com/repos/spack/spack/releases>`_ and search
+ for ``download_count`` to see this.
+
+
+.. _merging-releases-to-develop:
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Merging back into ``develop``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Once each release is complete, make sure that it is merged back into
+``develop`` with a merge commit:
+
+.. code-block:: console
+
+ $ git checkout develop
+ $ git merge --no-ff releases/v0.15
+ $ git push
+
+We merge back to ``develop`` because it:
+
+ * updates the version and ``CHANGELOG.md`` on ``develop``.
+ * ensures that your release tag is reachable from the head of
+ ``develop``
+
+We *must* use a real merge commit (via the ``--no-ff`` option) because it
+ensures that the release tag is reachable from the tip of ``develop``.
+This is necessary for ``spack -V`` to work properly -- it uses ``git
+describe --tags`` to find the last reachable tag in the repository and
+reports how far we are from it. For example:
+
+.. code-block:: console
+
+ $ spack -V
+ 0.14.2-1486-b80d5e74e5
+
+This says that we are at commit ``b80d5e74e5``, which is 1,486 commits
+ahead of the ``0.14.2`` release.
+
+We put this step last in the process because it's best to do it only once
+the release is complete and tagged. If you do it before you've tagged the
+release and later decide you want to tag some later commit, you'll need
+to merge again.
+
+
+.. _announcing-releases:
+
+^^^^^^^^^^^^^^^^^^^^
+Announcing a release
+^^^^^^^^^^^^^^^^^^^^
+
+We announce releases in all of the major Spack communication channels.
+Publishing the release takes care of GitHub. The remaining channels are
+Twitter, Slack, and the mailing list. Here are the steps:
+
+#. Make a tweet to announce the release. It should link to the release's
+ page on GitHub. You can base it on `this example tweet
+ <https://twitter.com/spackpm/status/1231761858182307840>`_.
+
+#. Ping ``@channel`` in ``#general`` on Slack (`spackpm.slack.com
+ <https://spackpm.slack.com>`_) with a link to the tweet. The tweet
+ will be shown inline so that you do not have to retype your release
+ announcement.
+
+#. Email the Spack mailing list to let them know about the release. As
+ with the tweet, you likely want to link to the release's page on
+ GitHub. It's also helpful to include some information directly in the
+ email. You can base yours on this `example email
+ <https://groups.google.com/forum/#!topic/spack/WT4CT9i_X4s>`_.
+
+Once you've announced the release, congratulations, you're done! You've
+finished making the release!
diff --git a/lib/spack/docs/getting_started.rst b/lib/spack/docs/getting_started.rst
index 226b1f0883..7b908465f5 100644
--- a/lib/spack/docs/getting_started.rst
+++ b/lib/spack/docs/getting_started.rst
@@ -818,7 +818,7 @@ Git
Some Spack packages use ``git`` to download, which might not work on
some computers. For example, the following error was
-encountered on a Macintosh during ``spack install julia-master``:
+encountered on a Macintosh during ``spack install julia@master``:
.. code-block:: console
diff --git a/lib/spack/docs/images/pr-commit.png b/lib/spack/docs/images/pr-commit.png
new file mode 100644
index 0000000000..a87c800ef5
--- /dev/null
+++ b/lib/spack/docs/images/pr-commit.png
Binary files differ
diff --git a/lib/spack/docs/images/projects.png b/lib/spack/docs/images/projects.png
new file mode 100644
index 0000000000..bd2971ff62
--- /dev/null
+++ b/lib/spack/docs/images/projects.png
Binary files differ