summaryrefslogtreecommitdiff
path: root/share
AgeCommit message (Collapse)AuthorFilesLines
2022-09-07Make GHA tests parallel by using xdist (#32361)Tom Scogland2-1/+6
* Add two no-op jobs named "all-prechecks" and "all" These are a suggestion from @tgamblin, they are stable named markers we can use from gitlab and possibly for required checks to make CI more resilient to refactors changing the names of specific checks. * Enable parallel testing using xdist for unit testing in CI * Normalize tmp paths to deal with macos * add -u flag compatibility to spack python As of now, it is accepted and ignored. The usage with xdist, where it is invoked specifically by `python -u spack python` which is then passed `-u` by xdist is the entire reason for doing this. It should never be used without explicitly passing -u to the executing python interpreter. * use spack python in xdist to support python 2 When running on python2, spack has many import cycles unless started through main. To allow that, this uses `spack python` as the interpreter, leveraging the `-u` support so xdist doesn't error out when it unconditionally requests unbuffered binary IO. * Use shutil.move to account for tmpdir being in a separate filesystem sometimes
2022-09-02Update bootstrap buildcache to v0.3 (#32262)Harmen Stoppels4-0/+44
This release allow to bootstrap patchelf from binaries.
2022-09-01Fix cause of checksum failures in public binary mirror (#32407)Scott Wittenburg2-4/+6
Move the copying of the buildcache to a root job that runs after all the child pipelines have finished, so that the operation can be coordinated across all child pipelines to remove the possibility of race conditions during potentially simlutandous copies. This lets us ensure the .spec.json.sig and .spack files for any spec in the root mirror always come from the same child pipeline mirror (though which pipeline is arbitrary). It also allows us to avoid copying of duplicates, which we now do.
2022-09-01New command, `spack change`, to change existing env specs (#31995)Peter Scheibel1-1/+10
If you have an environment like ``` $ cat spack.yaml spack: specs: [openmpi@4.1.0+cuda] ``` this PR provides a new command `spack change` that you can use to adjust environment specs from the command line: ``` $ spack change openmpi~cuda $ cat spack.yaml spack: specs: [openmpi@4.1.0~cuda] ``` in other words, this allows you to tweak the details of environment specs from the command line. Notes: * This is only allowed for environments that do not define matrices * This is possible but not anticipated to be needed immediately * If this were done, it should probably only be done for "named"/not-anonymous specs (i.e. we can change `openmpi+cuda` but not spec like `+cuda` or `@4.0.1~cuda`)
2022-09-01Fix CI for package only PRs (#32473)Massimiliano Culpo1-1/+1
2022-08-24oneapi e4s: use require: to force gcc build for some packages (#32309)Robert Cohn1-15/+24
2022-08-23spack ci: add support for running stand-alone tests (#27877)Tamara Dahlgren4-3/+53
This support requires adding the '--tests' option to 'spack ci rebuild'. Packages whose stand-alone tests are broken (in the CI environment) can be configured in gitlab-ci to be skipped by adding them to broken-tests-packages. Highlights include: - Restructured 'spack ci' help to provide better subcommand summaries; - Ensured only one InstallError (i.e., installer's) rather than allowing build_environment to have its own; and - Refactored CI and CDash reporting to keep CDash-related properties and behavior in a separate class. This allows stand-alone tests from `spack ci` to run when the `--tests` option is used. With `--tests`, stand-alone tests are run **after** a **successful** (re)build of the package. Test results are collected and report(able) using CDash. This PR adds the following features: - Adds `-t` and `--tests` to `spack ci rebuild` to run stand-alone tests; - Adds `--fail-fast` to stop stand-alone tests after the first failure; - Ensures a *single* `InstallError` across packages (i.e., removes second class from build environment); - Captures skipping tests for externals and uninstalled packages (for CDash reporting); - Copies test logs and outputs to the CI artifacts directory to facilitate debugging; - Parses stand-alone test results to report outputs from each `run_test` as separate test parts (CDash reporting); - Logs a test completion message to allow capture of timing of the last `run_test` part; - Adds the runner description to the CDash site to better distinguish entries in CDash tables; - Adds `gitlab-ci` `broken-tests-packages` to CI configuration to skip stand-alone testing for packages with known issues; - Changes `spack ci --help` so description of each subcommand is a single line; - Changes `spack ci <subcommand> --help` to provide the full description of each command (versus no description); and - Ensures `junit` test log file ends in an `.xml` extension (versus default where it does not). Tasks: - [x] Include the equivalent of the architecture information, or at least the host target, in the CDash output - [x] Upload stand-alone test results files as `test` artifacts - [x] Confirm tests are run in GitLab - [x] Ensure CDash results are uploaded as artifacts - [x] Resolve issues with CDash build-and test results appearing on same row of the table - [x] Add unit tests as needed - [x] Investigate why some (dependency) packages don't have test results (e.g., related from other pipelines) - [x] Ensure proper parsing and reporting of skipped tests (as `not run`) .. post- #28701 merge - [x] Restore the proper CDash URLand or mirror ONCE out-of-band testing completed
2022-08-22enable packages that use numpy for oneapi e4s stack (#32293)Robert Cohn1-3/+3
* enable geopm oneapi * enable packages that depend on py-numpy * disable nan warnings * undo geopm enable * undo geopm, it has multiple compile issues
2022-08-20spack list: add --tag flag (#32016)sparkyniner1-1/+1
* modified list.py and added functionality for --tag * Removed long and very long, shifted rest of code above return statement * removed results variable * added import statement at top * added the line accidentally deleted * added line accidentally deleted * changed p.name to p, added line inside if statement * line order switched * [@spackbot] updating style on behalf of sparkyniner * ran update completion command * add tests * Update lib/spack/spack/test/cmd/list.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * [@spackbot] updating style on behalf of sparkyniner * changed argument to mock_packages and moved code under filter by tag * removed bad rebase code and added additional test * [@spackbot] updating style on behalf of sparkyniner * added line removed earlier * added line removed earlier * replaced function * added more recommended changes Co-authored-by: sairaj <sairaj@sairajs-MacBook-Pro.local> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2022-08-19e4s %oneapi: add aml +ze (level zero) (#32161)eugeneswalker1-0/+1
2022-08-19e4s oneapi: compiler environment: prepend lib dir to LD_LIBRARY_PATH (#32227)eugeneswalker1-3/+7
2022-08-18e4s: add dealii +cuda (#32159)eugeneswalker1-0/+1
2022-08-18e4s %oneapi: add amrex +sycl (#32162)eugeneswalker1-1/+2
2022-08-18ascent: fix oneapi build (#32239)Robert Cohn1-1/+1
2022-08-18CI: move nightly macOS builds from GA to pipelines (#32231)Massimiliano Culpo1-0/+5
Modifications: - [x] Delete the `macOS builds nightly` workflow - [x] Removed shield in README.md - [x] Moved corresponding specs to `e4s-macos`
2022-08-17e4s mac ci: try lambda, the new mac studio runner (#32169)eugeneswalker2-7/+9
2022-08-17e4s oneapi stack: remove notes for now-fixed builds (#31839)eugeneswalker1-3/+1
2022-08-17CI: reduce the amount of tests run in the original concretizer (#32179)Massimiliano Culpo1-1/+4
* CI: reduce the amount of tests run in the original concretizer * Don't test Python 3.6 on the original concretizer
2022-08-16e4s: add flecsi +cuda (#32158)eugeneswalker1-0/+1
2022-08-16e4s: add ascent +cuda (#32153)eugeneswalker1-0/+1
2022-08-16e4s: add amrex +cuda (#32152)eugeneswalker1-1/+2
2022-08-16e4s: add papi +rocm (#32150)eugeneswalker1-0/+1
2022-08-16e4s: add caliper +rocm (#32149)eugeneswalker1-0/+1
2022-08-15unifyfs %oneapi: add CFLAGS=-Wno-unused-function to build environment (#32107)eugeneswalker1-2/+1
2022-08-15e4s ci: enable trilinos@13.4.0 +cuda build, issues resolved (#32124)eugeneswalker1-1/+1
2022-08-15e4s ci: enable hpctoolkit +rocm (#32126)eugeneswalker1-1/+1
2022-08-12e4s oneapi ci: enable hpx due to merged PR #32097 (#32117)eugeneswalker1-2/+1
2022-08-12e4s oneapi: build tau %oneapi ^binutils%gcc (#32106)eugeneswalker1-1/+1
2022-08-12gitlab ci: Make sure PR pipelines can push to buildcache (#32014)Scott Wittenburg1-0/+1
On PR pipelines we need to override the buildcache destination to point to the "spack-binaries-prs" bucket, otherwise, those pipelines try to push to the default mirror in a bucket for which they don't have write permission.
2022-08-12rocm: add v5.2 for the entire stack (#31591)renjithravindrankannath1-1/+3
2022-08-11Don't set `LD_LIBRARY_PATH` by default on Linux (#28354)Harmen Stoppels1-4/+4
`LD_LIBRARY_PATH` can break system executables (e.g., when an enviornment is loaded) and isn't necessary thanks to `RPATH`s. Packages that require `LD_LIBRARY_PATH` can set this in `setup_run_environment`. - [x] Prefix inspections no longer set `LD_LIBRARY_PATH` by default - [x] Document changes and workarounds for people who want `LD_LIBRARY_PATH`
2022-08-08axom@0.6.1 %oneapi: patch examples (#31991)eugeneswalker1-2/+1
* axom@0.6.1 %oneapi: patch examples * uncomment axom%oneapi ci build now that it is fixed
2022-08-08e4s oneapi ci: unify when possible (#31965)eugeneswalker1-1/+1
2022-08-03asp: refactor low level API to permit the injection of configurationMassimiliano Culpo1-1/+1
This allows writing extension commands that can benchmark different configurations in clingo, or try different configurations for a single test.
2022-08-03sundials@6.1.0:6.2.0 +rocm: patch nvector to use pic (#31910)eugeneswalker1-0/+1
* sundials@6.1.0:6.2.0 +rocm: patch nvector to use pic * e4s ci: add sundials +rocm
2022-08-02e4s oneapi ci: build vtk-m ~openmp due to issue #31830 (#31840)eugeneswalker1-2/+3
2022-08-02butterflypack %oneapi: patch CMakeLists to solve issue #31818 (#31848)eugeneswalker1-3/+2
* butterflypack %oneapi: patch CMakeLists to solve issue #31818 * uncomment builds affected by failing butterflypack Co-authored-by: e <e>
2022-08-01e4s oneapi stack: remove notes for now-fixed builds (#31839)eugeneswalker1-3/+1
2022-07-31style: simplify arguments with `--tool TOOL` and `--skip TOOL`Todd Gamblin1-1/+1
`spack style` tests were annoyingly brittle because we could not easily be specific about which tools to run (we had to use `--no-black`, `--no-isort`, `--no-flake8`, and `--no-mypy`). We should be able to specify what to run OR what to skip. Now you can run, e.g.: spack style --tool black,flake8 or: spack style --skip black,isort - [x] Remove `--no-black`, `--no-isort`, `--no-flake8`, and `--no-mypy` args. - [x] Add `--tool TOOL` argument. - [x] Add `--skip TOOL` argument. - [x] Allow either `--tool black --tool flake8` or `--tool black,flake8` syntax.
2022-07-31black: fix style check package and flake8 formatting for blackTodd Gamblin1-34/+5
Black will automatically fix a lot of the exceptions we previously allowed for directives, so we don't need them in our custom `flake8_formatter` anymore. - [x] remove `E501` (long line) exceptions for directives from `flake8_formatter`, as they won't help us now. - [x] Refine exceptions for long URLs in the `flake8_formatter`. - [x] Adjust the mock `flake8-package` to exhibit the exceptions we still allow. - [x] Update style tests for new `flake8-package`. - [x] Blacken style test.
2022-07-31black: configurationTodd Gamblin1-1/+1
This adds necessary configuration for flake8 and black to work together. This also sets the line length to 99, per the data here: * https://github.com/spack/spack/pull/24718#issuecomment-876933636 Given the data and the spirit of black's 88-character limit, we set the limit to 99 characters for all of Spack, because: * 99 is one less than 100, a nice round number, and all lines will fit in a 100-character wide terminal (even when the text editor puts a \ at EOL). * 99 is just past the knee the file size curve for packages, and it means that packages remain readable and not significantly longer than they are now. * It doesn't seem to hurt core -- files in core might change length by a few percent but seem like they'll be mostly the same as before -- just a bit more roomy. - [x] set line length to 99 - [x] remove most exceptions from `.flake8` and add the ones black cares about - [x] add `[tool.black]` to `pyproject.toml` - [x] make `black` run if available in `spack style --fix` Co-Authored-By: Tom Scogland <tscogland@llnl.gov>
2022-07-29e4s oneapi ci: uncomment pdt (#31803)eugeneswalker1-2/+6
* e4s oneapi ci: uncomment pdt * load oneapi compiler module before executing `spack ci rebuild`
2022-07-29e4s ci: add tasmanian +rocm (#31606)eugeneswalker1-0/+1
2022-07-29e4s oneapi ci: uncomment parallel-netcdf (#31804)eugeneswalker1-2/+1
2022-07-29e4s ci: add oneapi stack (#31781)eugeneswalker2-0/+503
* e4s ci: add oneapi stack * shorten padded_length to 256 * comment out pdt and add failure note
2022-07-27e4s ci: add slate +rocm (#31602)eugeneswalker1-0/+1
2022-07-26e4s ci stack: add spec: hdf5-vol-async (#31747)eugeneswalker1-0/+1
* e4s ci stack: add spec: hdf5-vol-async * hdf5-vol-async: add e4s tag
2022-07-20spack stage: add missing --fresh and --reuse (#31626)Harmen Stoppels1-1/+1
2022-07-18e4s ci: add ginkgo +rocm (#31603)eugeneswalker1-0/+1
2022-07-18containerize: fix missing environment activation (#31596)Massimiliano Culpo1-2/+1