diff options
Diffstat (limited to 'lib/spack/docs/developer_guide.rst')
| -rw-r--r-- | lib/spack/docs/developer_guide.rst | 367 |
1 files changed, 367 insertions, 0 deletions
diff --git a/lib/spack/docs/developer_guide.rst b/lib/spack/docs/developer_guide.rst index de2fe80f85..5c57f28226 100644 --- a/lib/spack/docs/developer_guide.rst +++ b/lib/spack/docs/developer_guide.rst @@ -495,3 +495,370 @@ The bottom of the output shows the top most time consuming functions, slowest on top. The profiling support is from Python's built-in tool, `cProfile <https://docs.python.org/2/library/profile.html#module-cProfile>`_. + +.. _releases: + +-------- +Releases +-------- + +This section documents Spack's release process. It is intended for +project maintainers, as the tasks described here require maintainer +privileges on the Spack repository. For others, we hope this section at +least provides some insight into how the Spack project works. + +.. _release-branches: + +^^^^^^^^^^^^^^^^ +Release branches +^^^^^^^^^^^^^^^^ + +There are currently two types of Spack releases: :ref:`major releases +<major-releases>` (``0.13.0``, ``0.14.0``, etc.) and :ref:`point releases +<point-releases>` (``0.13.1``, ``0.13.2``, ``0.13.3``, etc.). Here is a +diagram of how Spack release branches work:: + + o branch: develop (latest version) + | + o merge v0.14.1 into develop + |\ + | o branch: releases/v0.14, tag: v0.14.1 + o | merge v0.14.0 into develop + |\| + | o tag: v0.14.0 + |/ + o merge v0.13.2 into develop + |\ + | o branch: releases/v0.13, tag: v0.13.2 + o | merge v0.13.1 into develop + |\| + | o tag: v0.13.1 + o | merge v0.13.0 into develop + |\| + | o tag: v0.13.0 + o | + | o + |/ + o + +The ``develop`` branch has the latest contributions, and nearly all pull +requests target ``develop``. + +Each Spack release series also has a corresponding branch, e.g. +``releases/v0.14`` has ``0.14.x`` versions of Spack, and +``releases/v0.13`` has ``0.13.x`` versions. A major release is the first +tagged version on a release branch. Minor releases are back-ported from +develop onto release branches. This is typically done by cherry-picking +bugfix commits off of ``develop``. + +To avoid version churn for users of a release series, minor releases +should **not** make changes that would change the concretization of +packages. They should generally only contain fixes to the Spack core. + +Both major and minor releases are tagged. After each release, we merge +the release branch back into ``develop`` so that the version bump and any +other release-specific changes are visible in the mainline (see +:ref:`merging-releases-to-develop`). + + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Scheduling work for releases +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +We schedule work for releases by creating `GitHub projects +<https://github.com/spack/spack/projects>`_. At any time, there may be +several open release projects. For example, here are two releases (from +some past version of the page linked above): + +.. image:: images/projects.png + +Here, there's one release in progress for ``0.15.1`` and another for +``0.16.0``. Each of these releases has a project board containing issues +and pull requests. GitHub shows a status bar with completed work in +green, work in progress in purple, and work not started yet in gray, so +it's fairly easy to see progress. + +Spack's project boards are not firm commitments, and we move work between +releases frequently. If we need to make a release and some tasks are not +yet done, we will simply move them to next minor or major release, rather +than delaying the release to complete them. + +For more on using GitHub project boards, see `GitHub's documentation +<https://docs.github.com/en/github/managing-your-work-on-github/about-project-boards>`_. + +.. _major-releases: + +^^^^^^^^^^^^^^^^^^^^^ +Making Major Releases +^^^^^^^^^^^^^^^^^^^^^ + +Assuming you've already created a project board and completed the work +for a major release, the steps to make the release are as follows: + +#. Create two new project boards: + + * One for the next major release + * One for the next point release + +#. Move any tasks that aren't done yet to one of the new project boards. + Small bugfixes should go to the next point release. Major features, + refactors, and changes that could affect concretization should go in + the next major release. + +#. Create a branch for the release, based on ``develop``: + + .. code-block:: console + + $ git checkout -b releases/v0.15 develop + + For a version ``vX.Y.Z``, the branch's name should be + ``releases/vX.Y``. That is, you should create a ``releases/vX.Y`` + branch if you are preparing the ``X.Y.0`` release. + +#. Bump the version in ``lib/spack/spack/__init__.py``. See `this example from 0.13.0 + <https://github.com/spack/spack/commit/8eeb64096c98b8a43d1c587f13ece743c864fba9>`_ + +#. Updaate the release version lists in these files to include the new version: + + * ``lib/spack/spack/schema/container.py`` + * ``lib/spack/spack/container/images.json`` + + **TODO**: We should get rid of this step in some future release. + +#. Update ``CHANGELOG.md`` with major highlights in bullet form. Use + proper markdown formatting, like `this example from 0.15.0 + <https://github.com/spack/spack/commit/d4bf70d9882fcfe88507e9cb444331d7dd7ba71c>`_. + +#. Push the release branch to GitHub. + +#. Make sure CI passes on the release branch, including: + * Regular unit tests + * Build tests + * The E4S pipeline at `gitlab.spack.io <https://gitlab.spack.io>`_ + + If CI is not passing, submit pull requests to ``develop`` as normal + and keep rebasing the release branch on ``develop`` until CI passes. + +#. Follow the steps in :ref:`publishing-releases`. + +#. Follow the steps in :ref:`merging-releases-to-develop`. + +#. Follow the steps in :ref:`announcing-releases`. + + +.. _point-releases: + +^^^^^^^^^^^^^^^^^^^^^ +Making Point Releases +^^^^^^^^^^^^^^^^^^^^^ + +This assumes you've already created a project board for a point release +and completed the work to be done for the release. To make a point +release: + +#. Create one new project board for the next point release. + +#. Move any cards that aren't done yet to the next project board. + +#. Check out the release branch (it should already exist). For the + ``X.Y.Z`` release, the release branch is called ``releases/vX.Y``. For + ``v0.15.1``, you would check out ``releases/v0.15``: + + .. code-block:: console + + $ git checkout releases/v0.15 + +#. Cherry-pick each pull request in the ``Done`` column of the release + project onto the release branch. + + This is **usually** fairly simple since we squash the commits from the + vast majority of pull requests, which means there is only one commit + per pull request to cherry-pick. For example, `this pull request + <https://github.com/spack/spack/pull/15777>`_ has three commits, but + the were squashed into a single commit on merge. You can see the + commit that was created here: + + .. image:: images/pr-commit.png + + You can easily cherry pick it like this (assuming you already have the + release branch checked out): + + .. code-block:: console + + $ git cherry-pick 7e46da7 + + For pull requests that were rebased, you'll need to cherry-pick each + rebased commit individually. There have not been any rebased PRs like + this in recent point releases. + + .. warning:: + + It is important to cherry-pick commits in the order they happened, + otherwise you can get conflicts while cherry-picking. When + cherry-picking onto a point release, look at the merge date, + **not** the number of the pull request or the date it was opened. + + Sometimes you may **still** get merge conflicts even if you have + cherry-picked all the commits in order. This generally means there + is some other intervening pull request that the one you're trying + to pick depends on. In these cases, you'll need to make a judgment + call: + + 1. If the dependency is small, you might just cherry-pick it, too. + If you do this, add it to the release board. + + 2. If it is large, then you may decide that this fix is not worth + including in a point release, in which case you should remove it + from the release project. + + 3. You can always decide to manually back-port the fix to the release + branch if neither of the above options makes sense, but this can + require a lot of work. It's seldom the right choice. + +#. Bump the version in ``lib/spack/spack/__init__.py``. See `this example from 0.14.1 + <https://github.com/spack/spack/commit/ff0abb9838121522321df2a054d18e54b566b44a>`_. + +#. Updaate the release version lists in these files to include the new version: + + * ``lib/spack/spack/schema/container.py`` + * ``lib/spack/spack/container/images.json`` + + **TODO**: We should get rid of this step in some future release. + +#. Update ``CHANGELOG.md`` with a list of bugfixes. This is typically just a + summary of the commits you cherry-picked onto the release branch. See + `the changelog from 0.14.1 + <https://github.com/spack/spack/commit/ff0abb9838121522321df2a054d18e54b566b44a>`_. + +#. Push the release branch to GitHub. + +#. Make sure CI passes on the release branch, including: + * Regular unit tests + * Build tests + * The E4S pipeline at `gitlab.spack.io <https://gitlab.spack.io>`_ + + If CI does not pass, you'll need to figure out why, and make changes + to the release branch until it does. You can make more commits, modify + or remove cherry-picked commits, or cherry-pick **more** from + ``develop`` to make this happen. + +#. Follow the steps in :ref:`publishing-releases`. + +#. Follow the steps in :ref:`merging-releases-to-develop`. + +#. Follow the steps in :ref:`announcing-releases`. + + +.. _publishing-releases: + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Publishing a release on GitHub +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +#. Go to `github.com/spack/spack/releases + <https://github.com/spack/spack/releases>`_ and click ``Draft a new + release``. Set the following: + + * ``Tag version`` should start with ``v`` and contain *all three* + parts of the version, .g. ``v0.15.1``. This is the name of the tag + that will be created. + + * ``Target`` should be the ``releases/vX.Y`` branch (e.g., ``releases/v0.15``). + + * ``Release title`` should be ``vX.Y.Z`` (To match the tag, e.g., ``v0.15.1``). + + * For the text, paste the latest release markdown from your ``CHANGELOG.md``. + + You can save the draft and keep coming back to this as you prepare the release. + +#. When you are done, click ``Publish release``. + +#. Immediately after publishing, go back to + `github.com/spack/spack/releases + <https://github.com/spack/spack/releases>`_ and download the + auto-generated ``.tar.gz`` file for the release. It's the ``Source + code (tar.gz)`` link. + +#. Click ``Edit`` on the release you just did and attach the downloaded + release tarball as a binary. This does two things: + + #. Makes sure that the hash of our releases doesn't change over time. + GitHub sometimes annoyingly changes they way they generate + tarballs, and then hashes can change if you rely on the + auto-generated tarball links. + + #. Gets us download counts on releases visible through the GitHub + API. GitHub tracks downloads of artifacts, but *not* the source + links. See the `releases + page <https://api.github.com/repos/spack/spack/releases>`_ and search + for ``download_count`` to see this. + + +.. _merging-releases-to-develop: + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Merging back into ``develop`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Once each release is complete, make sure that it is merged back into +``develop`` with a merge commit: + +.. code-block:: console + + $ git checkout develop + $ git merge --no-ff releases/v0.15 + $ git push + +We merge back to ``develop`` because it: + + * updates the version and ``CHANGELOG.md`` on ``develop``. + * ensures that your release tag is reachable from the head of + ``develop`` + +We *must* use a real merge commit (via the ``--no-ff`` option) because it +ensures that the release tag is reachable from the tip of ``develop``. +This is necessary for ``spack -V`` to work properly -- it uses ``git +describe --tags`` to find the last reachable tag in the repository and +reports how far we are from it. For example: + +.. code-block:: console + + $ spack -V + 0.14.2-1486-b80d5e74e5 + +This says that we are at commit ``b80d5e74e5``, which is 1,486 commits +ahead of the ``0.14.2`` release. + +We put this step last in the process because it's best to do it only once +the release is complete and tagged. If you do it before you've tagged the +release and later decide you want to tag some later commit, you'll need +to merge again. + + +.. _announcing-releases: + +^^^^^^^^^^^^^^^^^^^^ +Announcing a release +^^^^^^^^^^^^^^^^^^^^ + +We announce releases in all of the major Spack communication channels. +Publishing the release takes care of GitHub. The remaining channels are +Twitter, Slack, and the mailing list. Here are the steps: + +#. Make a tweet to announce the release. It should link to the release's + page on GitHub. You can base it on `this example tweet + <https://twitter.com/spackpm/status/1231761858182307840>`_. + +#. Ping ``@channel`` in ``#general`` on Slack (`spackpm.slack.com + <https://spackpm.slack.com>`_) with a link to the tweet. The tweet + will be shown inline so that you do not have to retype your release + announcement. + +#. Email the Spack mailing list to let them know about the release. As + with the tweet, you likely want to link to the release's page on + GitHub. It's also helpful to include some information directly in the + email. You can base yours on this `example email + <https://groups.google.com/forum/#!topic/spack/WT4CT9i_X4s>`_. + +Once you've announced the release, congratulations, you're done! You've +finished making the release! |