diff options
author | Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com> | 2022-08-23 00:52:48 -0700 |
---|---|---|
committer | GitHub <noreply@github.com> | 2022-08-23 00:52:48 -0700 |
commit | 3c3b18d8588a82b6ace4ca5e852497f115e88479 (patch) | |
tree | 9ad8e683acb4556aa7eb0433615b6b07ab2dc331 /share/spack/spack-completion.bash | |
parent | 8b49790784456f997433c3ea245018d750d5ee2e (diff) | |
download | spack-3c3b18d8588a82b6ace4ca5e852497f115e88479.tar.gz spack-3c3b18d8588a82b6ace4ca5e852497f115e88479.tar.bz2 spack-3c3b18d8588a82b6ace4ca5e852497f115e88479.tar.xz spack-3c3b18d8588a82b6ace4ca5e852497f115e88479.zip |
spack ci: add support for running stand-alone tests (#27877)
This support requires adding the '--tests' option to 'spack ci rebuild'.
Packages whose stand-alone tests are broken (in the CI environment) can
be configured in gitlab-ci to be skipped by adding them to
broken-tests-packages.
Highlights include:
- Restructured 'spack ci' help to provide better subcommand summaries;
- Ensured only one InstallError (i.e., installer's) rather than allowing
build_environment to have its own; and
- Refactored CI and CDash reporting to keep CDash-related properties and
behavior in a separate class.
This allows stand-alone tests from `spack ci` to run when the `--tests`
option is used. With `--tests`, stand-alone tests are run **after** a
**successful** (re)build of the package. Test results are collected
and report(able) using CDash.
This PR adds the following features:
- Adds `-t` and `--tests` to `spack ci rebuild` to run stand-alone tests;
- Adds `--fail-fast` to stop stand-alone tests after the first failure;
- Ensures a *single* `InstallError` across packages
(i.e., removes second class from build environment);
- Captures skipping tests for externals and uninstalled packages
(for CDash reporting);
- Copies test logs and outputs to the CI artifacts directory to facilitate
debugging;
- Parses stand-alone test results to report outputs from each `run_test` as
separate test parts (CDash reporting);
- Logs a test completion message to allow capture of timing of the last
`run_test` part;
- Adds the runner description to the CDash site to better distinguish entries
in CDash tables;
- Adds `gitlab-ci` `broken-tests-packages` to CI configuration to skip
stand-alone testing for packages with known issues;
- Changes `spack ci --help` so description of each subcommand is a single line;
- Changes `spack ci <subcommand> --help` to provide the full description of
each command (versus no description); and
- Ensures `junit` test log file ends in an `.xml` extension (versus default where
it does not).
Tasks:
- [x] Include the equivalent of the architecture information, or at least the host target, in the CDash output
- [x] Upload stand-alone test results files as `test` artifacts
- [x] Confirm tests are run in GitLab
- [x] Ensure CDash results are uploaded as artifacts
- [x] Resolve issues with CDash build-and test results appearing on same row of the table
- [x] Add unit tests as needed
- [x] Investigate why some (dependency) packages don't have test results (e.g., related from other pipelines)
- [x] Ensure proper parsing and reporting of skipped tests (as `not run`) .. post- #28701 merge
- [x] Restore the proper CDash URLand or mirror ONCE out-of-band testing completed
Diffstat (limited to 'share/spack/spack-completion.bash')
-rwxr-xr-x | share/spack/spack-completion.bash | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/share/spack/spack-completion.bash b/share/spack/spack-completion.bash index 936593d98a..4cfde14e7a 100755 --- a/share/spack/spack-completion.bash +++ b/share/spack/spack-completion.bash @@ -612,7 +612,7 @@ _spack_ci_rebuild_index() { } _spack_ci_rebuild() { - SPACK_COMPREPLY="-h --help" + SPACK_COMPREPLY="-h --help -t --tests --fail-fast" } _spack_ci_reproduce_build() { |