summaryrefslogtreecommitdiff
path: root/.github
AgeCommit message (Collapse)AuthorFilesLines
2022-11-05Add support for Python 3.11 (#33505)Massimiliano Culpo3-10/+13
Argparse started raising ArgumentError exceptions when the same parser is added twice. Therefore, we perform the addition only if the parser is not there already Port match syntax to our unparser
2022-11-03Update metadata for bootstrapping (#33665)Massimiliano Culpo1-0/+6
2022-11-01Let pytest-cov create the xml directly (#33619)Massimiliano Culpo1-7/+1
`coverage` sometimes failed to combine, even if there were multiple reports.
2022-10-31Windows: fix library loading and enable Clingo bootstrapping (#33400)John W. Parent1-3/+3
Changes to improve locating shared libraries on Windows, which in turn enables the use of Clingo. This PR attempts to establish a proper distinction between linking on Windows vs. Linux/Mac: on Windows, linking is always done with .lib files (never .dll files). This somewhat complicates the model since the Spec.lib method could return libraries that were used for both linking and loading, but since these are not always the same on Windows, it was decided to treat Spec.libs as being for link-time libraries. Additional functions are added to help dependents locate run-time libraries. * Clingo is now the default concretizer on Windows * Clingo is now the concretizer used for unit tests on Windows * Fix a permissions issue that can occur while moving Git files during fetching/staging * Packages can now implement "win_add_library_dependent" to register files/directories that include libraries that would need to link to dependency dlls * Packages can now implement "win_add_rpath" to register the locations of dlls that dependents would want to load * "Spec.libs" on Windows is updated to return link-time libraries (i.e. .lib files, rather than .dll files) * PackageBase.rpath on Windows is now updated to return the most-likely locations where .dlls will be found (which is generally in the bin/ directory)
2022-10-29Deprecate spack bootstrap trust/untrust (#33600)Massimiliano Culpo3-11/+11
* Deprecate spack bootstrap trust/untrust * Update CI * Update tests
2022-10-27Update macOS Python version in CI to 3.10 (#33560)Massimiliano Culpo1-1/+1
2022-10-27Fix bootstrapping from sources in CI (#33538)Massimiliano Culpo1-6/+6
Since #32262 we are not testing bootstrapping from sources, since we didn't update the mirrors in tests
2022-10-25Update Spack Dockerfiles (#33500)Massimiliano Culpo1-1/+1
* Use spack bootstrap now in containers * Fix wrong path glob expression
2022-10-25Make CI on Windows fail fast (#33502)Massimiliano Culpo1-2/+2
2022-10-25Remove recursive symbolic link in lib/spack/docs from git repository (#33483)Massimiliano Culpo2-5/+0
Delete code removing the symlink during CI
2022-10-25CI: speed-up tests by dropping coverage on Python 2.7 (#33497)Massimiliano Culpo1-6/+6
2022-10-22Don't install xdist in CI on Python 2.7 (#33474)Massimiliano Culpo1-1/+6
2022-10-20FIX CI after git update (#33429)Massimiliano Culpo2-0/+8
Add `protocol.file.allow always` to git configuration in CI
2022-10-18build(deps): bump docker/setup-buildx-action from 2.1.0 to 2.2.0 (#33384)dependabot[bot]1-1/+1
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2.1.0 to 2.2.0. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](https://github.com/docker/setup-buildx-action/compare/95cb08cb2672c73d4ffd2f422e6d11953d2a9c70...c74574e6c82eeedc46366be1b0d287eff9085eb6) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-13Bump docker/login-action from 2.0.0 to 2.1.0 (#33268)dependabot[bot]1-2/+2
Bumps [docker/login-action](https://github.com/docker/login-action) from 2.0.0 to 2.1.0. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](https://github.com/docker/login-action/compare/49ed152c8eca782a232dede0303416e8f356c37b...f4ef78c080cd8ba55a85445d5b36e214a81df20a) --- updated-dependencies: - dependency-name: docker/login-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-13Bump docker/setup-buildx-action from 2.0.0 to 2.1.0 (#33267)dependabot[bot]1-1/+1
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2.0.0 to 2.1.0. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](https://github.com/docker/setup-buildx-action/compare/dc7b9719a96d48369863986a06765841d7ea23f6...95cb08cb2672c73d4ffd2f422e6d11953d2a9c70) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-11Bump actions/setup-python from 4.2.0 to 4.3.0 (#33166)dependabot[bot]4-12/+12
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.2.0 to 4.3.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/b55428b1882923874294fa556849718a1d7f2ca5...13ae5bb136fac2878aff31522b9efb785519f984) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-06build(deps): bump actions/checkout from 3.0.2 to 3.1.0 (#32998)dependabot[bot]7-25/+25
2022-09-20build(deps): bump codecov/codecov-action from 3.1.0 to 3.1.1 (#32717)dependabot[bot]3-7/+7
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 3.1.0 to 3.1.1. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md) - [Commits](https://github.com/codecov/codecov-action/compare/81cd2dc8148241f03f5839d295e000b8f761e378...d9f34f8cd5cb3b3eb79b3e4b5dae3a16df499a70) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-19Don't run bootstrap on package only PRs (#32689)Massimiliano Culpo1-2/+1
* Don't run bootstrap on package only PRs * Run bootstrap tests when ci.yaml is modified * Test a package only PR * Revert "Test a package only PR" This reverts commit af96b1af60b0c31efcc9a2875ffb1a809ef97241.
2022-09-16CI: add coverage on Windows (#32610)Massimiliano Culpo1-6/+21
Co-authored-by: Tom Scogland <scogland1@llnl.gov>
2022-09-13Add an issue template for reporting package stand-alone test failures (#32487)Tamara Dahlgren1-0/+62
2022-09-10ci: restore coverage computation (#32585)Massimiliano Culpo1-13/+15
* ci: restore coverage computation * Mark "test_foreground_background" as xfail * Mark "test_foreground_background_output" as xfail * Make number of processes explicit, remove verbosity on linux * Run coverage on just 3 Python jobs for linux * Run coverage on just 3 Python jobs for linux * Run coverage on just 2 Python jobs for linux * Add back verbose, since before we didn't encounter the xdist internal error * Reduce the workers to 2 * Try to use command line
2022-09-08ci: avoid running coverage on package only PRs (#32573)Massimiliano Culpo2-74/+11
* ci: remove !docs from "core" filters Written like it is now it causes package only PRs to run with coverage. * Try to skip job under condition, see if the workflow proceed * Try to cancel a running CI job * Simplify linux unit-tests, skip windows unit-tests on package PRs * Reduce the inputs to unit-tests workflow * Move control logic to main workflow, remove inputs * Revert "Move control logic to main workflow, remove inputs" This reverts commit 0c46fece4c49eb7a37585ec3ba651a31d7f958af. * Do not compute "with_coverage" since it's always == to "core" * Remove workflow dispatch from unit tests * Revert "Revert "Move control logic to main workflow, remove inputs"" This reverts commit dd4e4a4e61a825901e736348fd044d37e88c90b5. * Try to skip all from the main workflow * Add back bootstrap to needed checks for "all" * Restore the correct logic for conditionals
2022-09-07Make GHA tests parallel by using xdist (#32361)Tom Scogland7-42/+100
* Add two no-op jobs named "all-prechecks" and "all" These are a suggestion from @tgamblin, they are stable named markers we can use from gitlab and possibly for required checks to make CI more resilient to refactors changing the names of specific checks. * Enable parallel testing using xdist for unit testing in CI * Normalize tmp paths to deal with macos * add -u flag compatibility to spack python As of now, it is accepted and ignored. The usage with xdist, where it is invoked specifically by `python -u spack python` which is then passed `-u` by xdist is the entire reason for doing this. It should never be used without explicitly passing -u to the executing python interpreter. * use spack python in xdist to support python 2 When running on python2, spack has many import cycles unless started through main. To allow that, this uses `spack python` as the interpreter, leveraging the `-u` support so xdist doesn't error out when it unconditionally requests unbuffered binary IO. * Use shutil.move to account for tmpdir being in a separate filesystem sometimes
2022-09-02Reorder workflow execution in GHA (#32183)Tom Scogland6-244/+270
This patchset refactors our GitHub actions into a single top-level ci workflow that invokes a series of reusable actions. The main goal of this is to be able to easily control which tests run and in what order based on the success or failure of top-level prechecks. Our previous workflows ran in three sets: * nix tests: style and verification first, then linux and macos tests if successful * windows tests: style and verification first, then linux and macos tests if successful * bootstrap tests As a result, the bootstrap tests ran even if the style failed, and style and verification had to run on two different platforms despite running identical checks. I'm relatively sure that's because of the limitation on dependencies between steps in the jobs. Reusable workflows allow us to run the style, verification and now audit checks once, then depending on the results, and the files changed, run the appropriate nix, windows and bootstrap tests. While it saves only a few minutes by itself, this makes it easier to refactor checks to subset tests without having to replicate tests or other workflow components in the future. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-09-01Fix CI for package only PRs (#32473)Massimiliano Culpo1-1/+1
2022-08-18CI: move nightly macOS builds from GA to pipelines (#32231)Massimiliano Culpo1-71/+0
Modifications: - [x] Delete the `macOS builds nightly` workflow - [x] Removed shield in README.md - [x] Moved corresponding specs to `e4s-macos`
2022-08-17CI: reduce the amount of tests run in the original concretizer (#32179)Massimiliano Culpo1-2/+0
* CI: reduce the amount of tests run in the original concretizer * Don't test Python 3.6 on the original concretizer
2022-08-08build(deps): bump docker/build-push-action from 3.0.0 to 3.1.1 (#31964)dependabot[bot]1-1/+1
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3.0.0 to 3.1.1. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](https://github.com/docker/build-push-action/compare/e551b19e49efd4e98792db7592c17c09b89db8d8...c84f38281176d4c9cdb1626ffafcd6b3911b5d94) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-04Deprecate support for Python 3.5 (by removing it from CI) (#31908)Massimiliano Culpo3-7/+7
* Deprecate support for Python 3.5 (by removing it from CI) * Restore Python 3.9
2022-08-03Remove macOS 10.15 from bootstrap tests (#31890)Massimiliano Culpo1-1/+1
That version of the runners have been deprecated, see: https://github.blog/changelog/2022-07-20-github-actions-the-macos-10-15-actions-runner-image-is-being-deprecated-and-will-be-removed-by-8-30-22/
2022-08-03build(deps): bump actions/setup-python from 4.1.0 to 4.2.0 (#31882)dependabot[bot]4-19/+19
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.1.0 to 4.2.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/c4e89fac7e8767b327bbad6cb4d859eda999cf08...b55428b1882923874294fa556849718a1d7f2ca5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-31black: configurationTodd Gamblin2-3/+3
This adds necessary configuration for flake8 and black to work together. This also sets the line length to 99, per the data here: * https://github.com/spack/spack/pull/24718#issuecomment-876933636 Given the data and the spirit of black's 88-character limit, we set the limit to 99 characters for all of Spack, because: * 99 is one less than 100, a nice round number, and all lines will fit in a 100-character wide terminal (even when the text editor puts a \ at EOL). * 99 is just past the knee the file size curve for packages, and it means that packages remain readable and not significantly longer than they are now. * It doesn't seem to hurt core -- files in core might change length by a few percent but seem like they'll be mostly the same as before -- just a bit more roomy. - [x] set line length to 99 - [x] remove most exceptions from `.flake8` and add the ones black cares about - [x] add `[tool.black]` to `pyproject.toml` - [x] make `black` run if available in `spack style --fix` Co-Authored-By: Tom Scogland <tscogland@llnl.gov>
2022-07-12build(deps): bump actions/setup-python from 4.0.0 to 4.1.0 (#31526)dependabot[bot]4-19/+19
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.0.0 to 4.1.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/d09bd5e6005b175076f227b13d9730d56e9dcfcb...c4e89fac7e8767b327bbad6cb4d859eda999cf08) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-17py-setuptools: install setuptools from wheels directly (#31131)Massimiliano Culpo2-2/+2
When installing setuptools from sources in Spack, we might get into weird failures due to the way we use pip. In particular, for Spack it's necessary to install in a non-isolated pip environment to allow using PYTHONPATH as a selection method for all the build requirements of a Python package. This can fail when installing setuptools since there might be a setuptools version already installed for the Python interpreter being used, with different entry points than the one we want to install. Installing from wheels both pip and setuptools should harden our installation procedure in the context of: - Bootstrapping Python dependencies of Spack - Using external Python packages
2022-06-16CI: test binary bootstrapping of clingo on all macOS versions (#31160)Massimiliano Culpo1-1/+2
2022-06-13Pin setuptools version in Github Action Workflows (#31118)Massimiliano Culpo2-2/+2
fixes #31109
2022-06-10build(deps): bump actions/setup-python from 3.1.2 to 4 (#31059)dependabot[bot]4-19/+19
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3.1.2 to 4. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/98f2ad02fd48d057ee3b4d4f66525b231c3e52b6...d09bd5e6005b175076f227b13d9730d56e9dcfcb) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-09clingo: fix ~python build (#30824)Adam J. Stewart1-0/+1
2022-06-08Cancel running workflows automatically on PR update (#31044)Massimiliano Culpo5-0/+22
* Cancel running workflows automatically on PR update * Add the last update later to check cancellation is working * Use github.run_number instead of github.sha
2022-06-02Fix typo in GitHub Actions step name (#30924)Adam J. Stewart1-1/+1
2022-05-24buildcache: Update layout and signing (#30750)Scott Wittenburg1-0/+1
This PR introduces a new build cache layout and package format, with improvements for both efficiency and security. ## Old Format Currently a binary package consists of a `spec.json` file at the root and a `.spack` file, which is a `tar` archive containing a copy of the `spec.json` format, possibly a detached signature (`.asc`) file, and a tar-gzip compressed archive containing the install tree. ``` build_cache/ # metadata (for indexing) <arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.spec.json <arch>/ <compiler>/ <name>-<ver>/ # tar archive <arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.spack # tar archive contents: # metadata (contains sha256 of internal .tar.gz) <arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.spec.json # signature <arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.spec.json.asc # tar.gz-compressed prefix <arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.tar.gz ``` After this change, the nesting has been removed so that the `.spack` file is the compressed archive of the install tree. Now signed binary packages, will take the form of a clearsigned `spec.json` file (a `spec.json.sig`) at the root, while unsigned binary packages will contain a `spec.json` at the root. ## New Format ``` build_cache/ # metadata (for indexing, contains sha256 of .spack file) <arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.spec.json # clearsigned spec.json metadata <arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.spec.json.sig <arch>/ <compiler>/ <name>-<ver>/ # tar.gz-compressed prefix (may support more compression formats later) <arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.spack ``` ## Benefits The major benefit of this change is that the signatures on binary packages can be verified without: 1. Having to download the tarball, or 2. having to extract an unknown tarball. (1) is an improvement in efficiency; (2) is a security fix: we now ensure that we trust the binary before we try to run it through `tar`, which avoids potential attacks. ## Backward compatibility Also after this change, spack should still be able to handle the previous buildcache structure and binary mirrors with mixed layouts.
2022-05-21build(deps): bump actions/upload-artifact from 3 to 3.1.0 (#30778)dependabot[bot]2-3/+3
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3 to 3.1.0. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v3...3cea5372237819ed00197afe530f5a7ea3e805c8) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-19Run scheduled CI workflows only in the main repo (#30729)Wileam Y. Phan3-1/+16
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-05-10bootstrap: clean up CI workflows a bit (#30574)Todd Gamblin1-61/+66
- [x] Add `mkdir -p` and `chmod` to ensure `/home/spack-test` exists and has correct permissions. - [x] Remove version comments from dependabot-managed action commits - [x] Don't duplicate comment describing required fixes for distros with patched git
2022-05-09CI: fix bootstrapping on Fedora (#30559)Massimiliano Culpo1-0/+6
Fedora updated git, so we need to apply, again, the workaround for CVE-2022-24765
2022-05-06build(deps): bump docker/build-push-action from 2.10.0 to 3 (#30515)dependabot[bot]1-1/+1
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 2.10.0 to 3. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](https://github.com/docker/build-push-action/compare/ac9327eae2b366085ac7f6a2d02df8aa8ead720a...e551b19e49efd4e98792db7592c17c09b89db8d8) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-06build(deps): bump docker/setup-qemu-action from 1.2.0 to 2 (#30518)dependabot[bot]1-1/+1
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 1.2.0 to 2. - [Release notes](https://github.com/docker/setup-qemu-action/releases) - [Commits](https://github.com/docker/setup-qemu-action/compare/27d0a4f181a40b142cce983c5393082c365d1480...8b122486cedac8393e77aa9734c3528886e4a1a8) --- updated-dependencies: - dependency-name: docker/setup-qemu-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-06build(deps): bump docker/setup-buildx-action from 1.7.0 to 2 (#30517)dependabot[bot]1-1/+1
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 1.7.0 to 2. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](https://github.com/docker/setup-buildx-action/compare/f211e3e9ded2d9377c8cadc4489a4e38014bc4c9...dc7b9719a96d48369863986a06765841d7ea23f6) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>