diff options
author | Todd Gamblin <tgamblin@llnl.gov> | 2019-12-29 17:53:52 -0800 |
---|---|---|
committer | Todd Gamblin <tgamblin@llnl.gov> | 2020-01-01 21:37:02 -0800 |
commit | 4beb9fc5d36ab85cb5d2e43e451b26204f36cdcd (patch) | |
tree | 75b061a8c4bdbde82528e2113730b5772941321d /share | |
parent | de73121ebd9c2a64c269f129f42a7144d94cd245 (diff) | |
download | spack-4beb9fc5d36ab85cb5d2e43e451b26204f36cdcd.tar.gz spack-4beb9fc5d36ab85cb5d2e43e451b26204f36cdcd.tar.bz2 spack-4beb9fc5d36ab85cb5d2e43e451b26204f36cdcd.tar.xz spack-4beb9fc5d36ab85cb5d2e43e451b26204f36cdcd.zip |
tests: improved `spack test` command line options
Previously, `spack test` automatically passed all of its arguments to
`pytest -k` if no options were provided, and to `pytest` if they were.
`spack test -l` also provided a list of test filenames, but they didn't
really let you completely narrow down which tests you wanted to run.
Instead of trying to do our own weird thing, this passes `spack test`
args directly to `pytest`, and omits the implicit `-k`. This means we
can now run, e.g.:
```console
$ spack test spec_syntax.py::TestSpecSyntax::test_ambiguous
```
This wasn't possible before, because we'd pass the fully qualified name
to `pytest -k` and get an error.
Because `pytest` doesn't have the greatest ability to list tests, I've
tweaked the `-l`/`--list`, `-L`/`--list-long`, and `-N`/`--list-names`
options to `spack test` so that they help you understand the names
better. you can combine these options with `-k` or other arguments to do
pretty powerful searches.
This one makes it easy to get a list of names so you can run tests in
different orders (something I find useful for debugging `pytest` issues):
```console
$ spack test --list-names -k "spec and concretize"
cmd/env.py::test_concretize_user_specs_together
concretize.py::TestConcretize::test_conflicts_in_spec
concretize.py::TestConcretize::test_find_spec_children
concretize.py::TestConcretize::test_find_spec_none
concretize.py::TestConcretize::test_find_spec_parents
concretize.py::TestConcretize::test_find_spec_self
concretize.py::TestConcretize::test_find_spec_sibling
concretize.py::TestConcretize::test_no_matching_compiler_specs
concretize.py::TestConcretize::test_simultaneous_concretization_of_specs
spec_dag.py::TestSpecDag::test_concretize_deptypes
spec_dag.py::TestSpecDag::test_copy_concretized
```
You can combine any list option with keywords:
```console
$ spack test --list -k microarchitecture
llnl/util/cpu.py modules/lmod.py
```
```console
$ spack test --list-long -k microarchitecture
llnl/util/cpu.py::
test_generic_microarchitecture
modules/lmod.py::TestLmod::
test_only_generic_microarchitectures_in_root
```
Or just list specific files:
```console
$ spack test --list-long cmd/test.py
cmd/test.py::
test_list test_list_names_with_pytest_arg
test_list_long test_list_with_keywords
test_list_long_with_pytest_arg test_list_with_pytest_arg
test_list_names
```
Hopefully this stuff will help with debugging test issues.
- [x] make `spack test` send args directly to `pytest` instead of trying
to do fancy things.
- [x] rework `--list`, `--list-long`, and add `--list-names` to make
searching for tests easier.
- [x] make it possible to mix Spack's list args with `pytest` args
(they're just fancy parsing around `pytest --collect-only`)
- [x] add docs
- [x] add tests
- [x] update spack completion
Diffstat (limited to 'share')
-rwxr-xr-x | share/spack/spack-completion.bash | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/share/spack/spack-completion.bash b/share/spack/spack-completion.bash index ee2eaa8286..2836db1896 100755 --- a/share/spack/spack-completion.bash +++ b/share/spack/spack-completion.bash @@ -1072,7 +1072,8 @@ function _spack_test { if $list_options then compgen -W "-h --help -H --pytest-help -l --list - -L --long-list" -- "$cur" + -L --list-long -N --list-names -s -k + --showlocals" -- "$cur" else compgen -W "$(_tests)" -- "$cur" fi |