summaryrefslogtreecommitdiff
path: root/share/spack/qa/run-unit-tests
AgeCommit message (Collapse)AuthorFilesLines
2020-03-20Recover coverage from subprocesses during unit tests (#15354)Tamara Dahlgren1-6/+2
* Recover coverage from subprocesses during unit tests
2020-01-22Use `spack commands --format=bash` to generate shell completion (#14393)Adam J. Stewart1-7/+10
This PR adds a `--format=bash` option to `spack commands` to auto-generate the Bash programmable tab completion script. It can be extended to work for other shells. Progress: - [x] Fix bug in superclass initialization in `ArgparseWriter` - [x] Refactor `ArgparseWriter` (see below) - [x] Ensure that output of old `--format` options remains the same - [x] Add `ArgparseCompletionWriter` and `BashCompletionWriter` - [x] Add `--aliases` option to add command aliases - [x] Standardize positional argument names - [x] Tests for `spack commands --format=bash` coverage - [x] Tests to make sure `spack-completion.bash` stays up-to-date - [x] Tests for `spack-completion.bash` coverage - [x] Speed up `spack-completion.bash` by caching subroutine calls This PR also necessitates a significant refactoring of `ArgparseWriter`. Previously, `ArgparseWriter` was mostly a single `_write` method which handled everything from extracting the information we care about from the parser to formatting the output. Now, `_write` only handles recursion, while the information extraction is split into a separate `parse` method, and the formatting is handled by `format`. This allows subclasses to completely redefine how the format will appear without overriding all of `_write`. Co-Authored-by: Todd Gamblin <tgamblin@llnl.gov>
2019-12-30copyright: update copyright dates for 2020 (#14328)Todd Gamblin1-1/+1
2019-12-16Travis exits at the first failing test, pin codecov at v4.5.4 (#14179)Massimiliano Culpo1-1/+1
Before this commit we used to run the entire unit test suite in the presence of a failure. Since we currently rely a lot on the state of the filesystem etc. the end report was most of the time showing spurious failures that were a consequence of the first failing test. This PR makes unit tests exit at the first failing test Also, pin codecov at v4.5.4 (last one supporting Python 2.6)
2019-07-05tests: add tests for setup-env.shTodd Gamblin1-0/+30
- tests use a shell-script harness and test all Spack commands that require special shell support. - tests work in bash, zsh, and dash - run setup-env.sh tests on macos and linux builds. - we run them on macos and linux
2019-06-08tests: restore `spack spec mpileaks` in test output for Python 2 (#11664)Todd Gamblin1-5/+3
- also remove coverage calls of these commands - we JUST want coverage to tell us what is in the test suite.
2019-06-05travis: disable `spack spec` for python 2.7 as well as 2.6Todd Gamblin1-1/+1
- keep this disabled until the `compiler find` fork bomb is fixed.
2019-05-10bugfix: Do not run spack spec in the test environment for Python 2.6Todd Gamblin1-1/+3
- Trying to get our Python 2.6 tests working again. - This seems to crash travis in the 2.6 environment, for whatever reason.
2019-01-01copyright: update license headers for 2013-2019 copyright.Todd Gamblin1-1/+1
2018-10-17relicense: replace LGPL headers with Apache-2.0/MIT SPDX headersTodd Gamblin1-0/+6
- remove the old LGPL license headers from all files in Spack - add SPDX headers to all files - core and most packages are (Apache-2.0 OR MIT) - a very small number of remaining packages are LGPL-2.1-only
2018-09-24travis: require sudo in an attempt to avoid build timeouts (#9327)Todd Gamblin1-1/+5
- Many container builds are timing out frequently during Spack tests in Travis CI. - Travis recommends to try `sudo: required` to see whether this is an infrastructure issue or something else. - added `sudo: required` to all Linux builds. - added --verbose to `spack test` invocation so that we can see more easily what tests it's timing out on. Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
2018-04-12Fixes conflicts in OSX nightly tests (#7732)Massimiliano Culpo1-1/+1
fixes #7593 Unit tests on OSX are trying to concretize mpileaks, and they fail due to a conflict in the package: "%gcc@7.2.0:" conflicts with "elfutils@0.163" This solves the issue asking explicitly to concretize against elfutils@1.170
2018-03-29update tests: concretize mpileaks with gcc (#7635)scheibelp1-1/+1
Fixes #7593 By default MacOS concretizes using the clang compiler. The unit tests include a call to "spack spec mpileaks", which has elfutils as a dependency; #7096 added a conflict in elfutils to avoid building with clang, which lead to the MacOS unit tests to start failing. This updates the concretization to force using gcc when concretizing mpileaks.
2018-01-20Restore multiprocessing in unit tests (#6949)Massimiliano Culpo1-1/+0
* Revert "Travis: use --concurrency=multiprocessing only on build tests (#6872)" This reverts commit 596d463714d46929f81db62b171c5a772679bb12. * Removing 'coverage combine' in test script According to what was discovered in #6887, one of the problems is calling 'coverage combine' twice without the '-a' flag. This removes the first call within our test scripts.
2017-05-08rework spack help (#3033)Todd Gamblin1-0/+4
- Full help is now only generated lazily, when needed. - Executing specific commands doesn't require loading all of them. - All commands are only loaded if we need them for help. - There is now short and long help: - short help (spack help) shows only basic spack options - long help (spack help -a) shows all spack options - Both divide help on commands into high-level sections - Commands now specify attributes from which help is auto-generated: - description: used in help to describe the command. - section: help section - level: short or long - Clean up command descriptions - Add a `spack docs` command to open full documentation in the browser. - move `spack doc` command to `spack pydoc` for clarity - Add a `spack --spec` command to show documentation on the spec syntax.
2017-05-01Allow user to specify profile sort column on the command line. (#4056)Todd Gamblin1-0/+3
- Add -P <STAT> argument so that caller can specify a sort column for cProfile. Can specify multiple columns with commas. e.g.: spack -P cumtime,module - Add --lines option to Spack spec to control number of profile lines displayed - Sort by time by default (because it works in all Python versions) - Show sort column options in command help. - Do a short profile run in the unit tests.
2017-04-27Separate integration tests; simplify test scripts (#4006)Todd Gamblin1-34/+6
* Separate build integration tests; simplify test scripts - Move build tests out of the regular Travis unit tests, add more smoke test packages to build. - Run all test scripts with bash -e, which fails on error. - Factor coverage out into a Travis environment variable, so it's more obvious from .travis.yml which tests contribute to coverage and which don't. - Factor dependency checking and much of the front-matter in tests scripts into a setup.sh script, which is sourced by all the test scripts. Extra cruft in each tests script now reduced to 2 lines at the beginning.
2017-04-21Coverage for multiple Python versions. (#3951)Todd Gamblin1-1/+3
Update tests to use codecov for multiple python versions.
2017-01-01tests: `coverage combine` run only if tests succeeded (#2712)Massimiliano Culpo1-2/+1
2016-12-30Run coverage for the `spack install` command as well as `spack test`.Todd Gamblin1-1/+2
- Also don't run coverage on OSX.
2016-12-29unit tests: replace nose with pytest (#2502)Massimiliano Culpo1-1/+6
* Porting: substitute nose with ytest This huge commit substitutes nose with pytest as a testing system. Things done here: * deleted external/nose as it is no longer used * moved mock resources in their own directory 'test/mock/' * ported two tests (cmd/find, build_system) to pytest native syntax as an example * build_environment, log: used monkeypatch instead of try/catch * moved global mocking of fetch_cache to an auto-used fixture * moved global mocking from test/__init__.py to conftest.py * made `spack test` a wrapper around pytest * run-unit-tests: avoid running python 2.6 tests under coverage to speed them up * use `pytest --cov` instead of coverage run to cut down testing time * mock/packages_test: moved mock yaml configuration to files instead of leaving it in the code as string literals * concretize.py: ported tests to native pytest, reverted multiprocessing in pytest.ini as it was creating the wrong report for coveralls * conftest.py, fixtures: added docstrings * concretize_preferences.py: uses fixtures instead of subclassing MockPackagesTest * directory_layout.py: uses fixtures instead of subclassing MockPackagesTest * install.py: uses fixtures instead of subclassing MockPackagesTest * optional_deps.py: uses fixtures instead of subclassing MockPackagesTest optional_deps.py: uses fixtures instead of subclassing MockPackagesTest * packages.py: uses fixtures instead of subclassing MockPackagesTest * provider_index.py: uses fixtures instead of subclassing MockPackagesTest * spec_yaml.py: uses fixtures instead of subclassing MockPackagesTest * multimethod.py: uses fixtures instead of subclassing MockPackagesTest * install.py: now uses mock_archive_url * git_fetch.py: uses fixtures instead of subclassing MockPackagesTest * hg_fetch.py: uses fixtures instead of subclassing MockPackagesTest * svn_fetch.py, mirror.py: uses fixtures instead of subclassing MockPackagesTest repo.py: deleted * test_compiler_cmd.py: uses fixtures instead of subclassing MockPackagesTest * cmd/module.py, cmd/uninstall.py: uses fixtures instead of subclassing MockDatabase * database.py: uses fixtures instead of subclassing MockDatabase, removed mock/database * pytest: uncluttering fixture implementations * database: changing the scope to 'module' * config.py: uses fixtures instead of subclassing MockPackagesTest * spec_dag.py, spec_semantics.py: uses fixtures instead of subclassing MockPackagesTest * stage.py: uses fixtures instead of subclassing MockPackagesTest. Removed mock directory * pytest: added docstrings to all the fixtures * pytest: final cleanup * build_system_guess.py: fixed naming and docstrings as suggested by @scheibelp * spec_syntax.py: added expected failure on parsing multiple specs closes #1976 * Add pytest and pytest-cov to Spack externals. * Make `spack flake8` ignore externals. * run-unit-tests runs spack test and not pytest. * Remove all the special stuff for `spack test` - Remove `conftest.py` magic and all the special case stuff in `bin/spack` - Spack commands can optionally take unknown arguments, if they want to handle them. - `spack test` is now a command like the others. - `spack test` now just delegates its arguments to `pytest`, but it does it by receiving unknown arguments and NOT taking an explicit help argument. * Fix error in fixtures. * Improve `spack test` command a bit. - Now supports an approximation of the old simple interface - Also supports full pytest options if you want them. * Use external coverage instead of pytest-cov * Make coverage use parallel-mode. * change __init__.py docs to include pytest
2016-08-30Always run spack unit testsAdam J. Stewart1-21/+0
2016-08-30Added more missing dependenciesAdam J. Stewart1-1/+3
2016-08-30Add mercurial package, used as test dependencyAdam J. Stewart1-1/+6
2016-08-30Only run unit tests when core Spack framework is modifiedAdam J. Stewart1-8/+48
2016-07-30qa : fixes #1370 (no flake8 on python 2.6, coveralls only python 2.7)alalazo1-0/+20