summaryrefslogtreecommitdiff
path: root/.github
AgeCommit message (Collapse)AuthorFilesLines
2021-07-24`spack style`: automatically bootstrap dependenciesvsoch1-2/+1
This uses our bootstrapping logic to automatically install dependencies for `spack style`. Users should no longer have to pre-install all of the tools (`isort`, `mypy`, `black`, `flake8`). The command will do it for them. - [x] add logic to bootstrap specs with specific version requirements in `spack style` - [x] remove style tools from CI requirements (to ensure we test bootstrapping) - [x] rework dependencies for `mypy` and `py-typed-ast` - `py-typed-ast` needs to be a link dependency - it needs to be at 1.4.1 or higher to work with python 3.9 Signed-off-by: vsoch <vsoch@users.noreply.github.com>
2021-07-24Bump codecov/action to v2.0.2 (#24983)Massimiliano Culpo1-5/+8
* build(deps): bump codecov/codecov-action from 1 to 2.0.1 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1 to 2.0.1. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md) - [Commits](https://github.com/codecov/codecov-action/compare/v1...v2.0.1) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Update arguments to codecov action * Try to delete the symbolic link to root folder Hopefully this should get rid of the ELOOP error * Delete also for shell tests and MacOS * Bump to v2.0.2 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-09coverage: move config from `.coveragerc` to `pyproject.toml`Todd Gamblin1-4/+4
Getting rid of another top-level file. `coverage.py` has supported `pyproject.toml` since version 5.0, and all versions of coverage so far work with python 2.7. We just need to ensure that a version of coverage with the `toml` extra is installed in the test environment. I tested this with `coverage run`, `coverage report`, and `coverage html`.
2021-07-09mypy: move configuration to pyproject.toml (#24802)Todd Gamblin1-2/+2
This moves our `mypy` configuration from `.mypy.ini` to `.pyproject.toml` and increases the minimum `mypy` version in the tests. - [x] move `mypy` configuration to `pyproject.toml` - [x] remove `.mypy.ini` - [x] ensure that `mypy` version .900 or higher is used in tests
2021-07-08imports: sort imports everywhere in Spack (#24695)Todd Gamblin1-1/+1
* fix remaining flake8 errors * imports: sort imports everywhere in Spack We enabled import order checking in #23947, but fixing things manually drives people crazy. This used `spack style --fix --all` from #24071 to automatically sort everything in Spack so PR submitters won't have to deal with it. This should go in after #24071, as it assumes we're using `isort`, not `flake8-import-order` to order things. `isort` seems to be more flexible and allows `llnl` mports to be in their own group before `spack` ones, so this seems like a good switch.
2021-07-07cvs tests: don't use dateutil at allTodd Gamblin1-6/+5
`dateutil.parser` was an optional dependency for CVS tests. It was failing on macOS beacuse the dateutil types were not being installed, and mypy was failing *even when the CVS tests were skipped*. This seems like it was an oversight on macOS -- `types-dateutil-parser` was not installed there, though it was on Linux unit tests. It takes 6 lines of YAML and some weird test-skipping logic to get `python-dateutil` and `types-python-dateutil` installed in all the tests where we need them, but it only takes 4 lines of code to write the date parser we need for CVS, so I just did that instead. Note that CVS date format can vary from system to system, but it seems like it's always pretty similar for the parts we care about. - [x] Replace dateutil.parser with a simpler date regex - [x] Lose the dependency on `dateutil.parser`
2021-07-07style: Move isort configuration to pyproject.tomlTodd Gamblin1-2/+2
- [x] Remove flake8-import-order checks, as we only need isort for this - [x] Clean up configuration and requirements
2021-07-07style: add support for `isort` and `--fix`Danny McClanahan1-2/+2
2021-07-04Remove add-maintainers-as-reviewers action (#24700)Adam J. Stewart2-91/+0
2021-06-29vermin: show line numbers of violations (#24580)Massimiliano Culpo1-2/+2
This commit runs vermin with the --violations option that shows details of the violations to target requirements.
2021-06-28Use flake8-import-order to enforce PEP-8 compliance (#23947)Adam J. Stewart1-2/+2
2021-06-23Fix broken CI for package only PRs, make dateutil not strictly required (#24484)Massimiliano Culpo1-0/+4
* Force the Python interpreter with an env variable This commit forces the Python interpreter with an environment variable, to ensure that the Python set by the "setup-python" action is the one being used. Due to the policy adopted by Spack to prefer python3 over python we may end up picking a Python 3.X interpreter where Python 2.7 was meant to be used. * Revert "Update conftest.py (#24473)" This reverts commit 477c8ce8205ec149fa897c9d83e530815c978d8b. * Make python-dateutil a soft dependency for unit tests Before #23212 people could clone spack and run ``` spack unit-tests ``` while now this is not possible, since python-dateutil is a required but not vendored dependency. This change makes it not a hard requirement, i.e. it will be used if found in the current interpreter. * Workaround mypy complaint
2021-06-22Implement CVS fetcher (#23212)Erik Schnetter1-10/+13
Spack packages can now fetch versions from CVS repositories. Note this fetch mechanism is unsafe unless using :extssh:. Most public CVS repositories use an insecure protocol implemented as part of CVS.
2021-06-08mypy: add types-six to the list of installed packages (#24207)Massimiliano Culpo1-1/+1
See https://mypy-lang.blogspot.com/2021/05/the-upcoming-switch-to-modular-typeshed.html for a broader explanation.
2021-05-28build(deps): bump actions/cache from 2.1.5 to 2.1.6 (#23983)dependabot[bot]1-1/+1
Bumps [actions/cache](https://github.com/actions/cache) from 2.1.5 to 2.1.6. - [Release notes](https://github.com/actions/cache/releases) - [Commits](https://github.com/actions/cache/compare/v2.1.5...v2.1.6) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-22Spack can Use RHEL8's `platform-python` if nothing else is available. (#23857)Todd Gamblin1-2/+0
This adds RHEL8's `/usr/libexec/platform-python` to Spack's list of preferred pythons. It will only be used if no other `python` is available in the `PATH`. We have been testing with this python for a while now, and it seems to do all that we need. If Spack one day isn't able to work with it, we'll take it out, but for now it is useful to allow Spack to be used on RHEL8 without a dedicated `python` installation.
2021-05-14build tests: put an upper bound on the version of GCC being used (#23630)Massimiliano Culpo1-1/+1
2021-05-13build(deps): bump actions/cache from 2.1.4 to 2.1.5 (#23584)dependabot[bot]1-1/+1
Bumps [actions/cache](https://github.com/actions/cache) from 2.1.4 to 2.1.5. - [Release notes](https://github.com/actions/cache/releases) - [Commits](https://github.com/actions/cache/compare/v2.1.4...v2.1.5) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-04-22qa: install clingo-cffi from pip (#23163)Massimiliano Culpo1-23/+54
Clingo has been released on PyPI, so there are no more concerns on our CI depending on pypy.test for installing the wheel. Apparently we have parts of Spack which are not compatible with kcov > 3.4
2021-03-20QA: don't run build tests on each commit (#22430)Massimiliano Culpo1-0/+12
This applies the same rules on push to develop that we use for PRs
2021-03-20QA: reduce number of unit tests for jobs not in the matrix (#22426)Massimiliano Culpo1-7/+28
* QA: reduce number of unit tests for jobs not in the matrix * Fixup for CentOS6 dependencies * Put correct conditions back in place * Add dependency on changes
2021-03-19CI: treat push to develop in the same way as PRs (#22421)Massimiliano Culpo1-5/+3
2021-03-19CI: drastically reduce the number of tests for package only PRs (#22410)Massimiliano Culpo1-1/+1
PRs that change only package recipes will only run tests under "package_sanity.py" and without coverage. This should result in a huge drop the cpu-time spent in CI for most PRs.
2021-03-16The action to detect changes needs a repository to be checked out on push ↵Massimiliano Culpo1-0/+4
event (#22324)
2021-03-16Speed-up CI by reorganizing tests (#22247)Massimiliano Culpo4-283/+337
* unit tests: mark slow tests as "maybeslow" This commit also removes the "network" marker and marks every "network" test as "maybeslow". Tests marked as db are maintained, but they're not slow anymore. * GA: require style tests to pass before running unit-tests * GA: make MacOS unit tests fail fast * GA: move all unit tests into the same workflow, run style tests as a prerequisite All the unit tests have been moved into the same workflow so that a single run of the dorny/paths-filter action can be used to ask for coverage based on the files that have been changed in a PR. The basic idea is that for PRs that introduce only changes to packages coverage is not necessary, this resulting in a faster execution of the tests. Also, for package only PRs slow unit tests are skipped. Finally, MacOS and linux unit tests are now conditional on style tests passing meaning that e.g. we won't waste a MacOS worker if we know that the PR has flake8 issues. * Addressed review comments * Skipping slow tests on MacOS for package only recipes * QA: make tests on changes correct before merging
2021-03-03Bootstrap clingo from sources (#21446)Massimiliano Culpo1-23/+9
* Allow the bootstrapping of clingo from sources Allow python builds with system python as external for MacOS * Ensure consistent configuration when bootstrapping clingo This commit uses context managers to ensure we can bootstrap clingo using a consistent configuration regardless of the use case being managed. * Github actions: test clingo with bootstrapping from sources * Add command to inspect and clean the bootstrap store Prevent users to set the install tree root to the bootstrap store * clingo: documented how to bootstrap from sources Co-authored-by: Gregory Becker <becker33@llnl.gov>
2021-02-24Run clingo-cffi tests in a container (#21913)Massimiliano Culpo1-36/+6
There clingo-cffi job has two issues to be solved: 1. It uses the default concretizer 2. It requires a package from https://test.pypi.org/simple/ The former can be fixed by setting the SPACK_TEST_SOLVER environment variable to "clingo". The latter though requires clingo-cffi to be pushed to a more stable package index (since https://test.pypi.org/simple/ is meant as a scratch version of PyPI that can be wiped at any time). For the time being run the tests in a container. Switch back to PyPI whenever a new official version of clingo will be released.
2021-02-23Updates to support clingo-cffi (#20657)Josh Essman1-0/+51
* Support clingo when used with cffi Clingo recently merged in a new Python module option based on cffi. Compatibility with this module requires a few changes to spack - it does not automatically convert strings/ints/etc to Symbol and clingo.Symbol.string throws on failure. manually convert str/int to clingo.Symbol types catch stringify exceptions add job for clingo-cffi to Spack CI switch to potassco-vendored wheel for clingo-cffi CI on_unsat argument when cffi
2021-02-16Add RHEL8 Universal Base Image with platform-python to CI unit tests (#21655)Chuck Atkins1-0/+27
2021-02-08build(deps): bump actions/cache from v2 to v2.1.4 (#21529)dependabot[bot]1-1/+1
Bumps [actions/cache](https://github.com/actions/cache) from v2 to v2.1.4. - [Release notes](https://github.com/actions/cache/releases) - [Commits](https://github.com/actions/cache/compare/v2...26968a09c0ea4f3e233fdddbafd1166051a095f6) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-02-04py-mypy: add v0.800 (#21386)Adam J. Stewart1-1/+1
2021-01-06qa: use dependabot to update Github ActionsMassimiliano Culpo1-0/+7
2021-01-02copyrights: update all files with license headers for 2021Todd Gamblin1-1/+1
- [x] add `concretize.lp`, `spack.yaml`, etc. to licensed files - [x] update all licensed files to say 2013-2021 using `spack license update-copyright-year` - [x] appease mypy with some additions to package.py that needed for oneapi.py
2020-12-22add mypy to style checks; rename `spack flake8` to `spack style` (#20384)Tom Scogland2-7/+7
I lost my mind a bit after getting the completion stuff working and decided to get Mypy working for spack as well. This adds a `.mypy.ini` that checks all of the spack and llnl modules, though not yet packages, and fixes all of the identified missing types and type issues for the spack library. In addition to these changes, this includes: * rename `spack flake8` to `spack style` Aliases flake8 to style, and just runs flake8 as before, but with a warning. The style command runs both `flake8` and `mypy`, in sequence. Added --no-<tool> options to turn off one or the other, they are on by default. Fixed two issues caught by the tools. * stub typing module for python2.x We don't support typing in Spack for python 2.x. To allow 2.x to support `import typing` and `from typing import ...` without a try/except dance to support old versions, this adds a stub module *just* for python 2.x. Doing it this way means we can only reliably use all type hints in python3.7+, and mypi.ini has been updated to reflect that. * add non-default black check to spack style This is a first step to requiring black. It doesn't enforce it by default, but it will check it if requested. Currently enforcing the line length of 79 since that's what flake8 requires, but it's a bit odd for a black formatted project to be quite that narrow. All settings are in the style command since spack has no pyproject.toml and I don't want to add one until more discussion happens. Also re-format `style.py` since it no longer passed the black style check with the new length. * use style check in github action Update the style and docs action to use `spack style`, adding in mypy and black to the action even if it isn't running black right now.
2020-11-18spack test (#15702)Greg Becker2-2/+2
Users can add test() methods to their packages to run smoke tests on installations with the new `spack test` command (the old `spack test` is now `spack unit-test`). spack test is environment-aware, so you can `spack install` an environment and then run `spack test run` to run smoke tests on all of its packages. Historical test logs can be perused with `spack test results`. Generic smoke tests for MPI implementations, C, C++, and Fortran compilers as well as specific smoke tests for 18 packages. Inside the test method, individual tests can be run separately (and continue to run best-effort after a test failure) using the `run_test` method. The `run_test` method encapsulates finding test executables, running and checking return codes, checking output, and error handling. This handles the following trickier aspects of testing with direct support in Spack's package API: - [x] Caching source or intermediate build files at build time for use at test time. - [x] Test dependencies, - [x] packages that require a compiler for testing (such as library only packages). See the packaging guide for more details on using Spack testing support. Included is support for package.py files for virtual packages. This does not change the Spack interface, but is a major change in internals. Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov> Co-authored-by: wspear <wjspear@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-11-17Github actions: add CI for ASP based solverMassimiliano Culpo1-0/+21
2020-11-12run unit tests on 3.8 only for Mac OS vs. both 3.8 and 3.9 (#19889)Peter Scheibel1-1/+1
2020-11-12macos: update build process to use spawn instead of fork (#18205)Peter Scheibel1-1/+4
Spack creates a separate process to do package installation. Different operating systems and Python versions use different methods to create it but up until Python 3.8 both Linux and Mac OS used "fork" (which duplicates process memory, file descriptor table, etc.). Python >= 3.8 on Mac OS prefers creating an entirely new process (referred to as the "spawn" start method) because "fork" was found to cause issues (in other words "spawn" is the default start method used by multiprocessing.Process). Spack was dependent on the particular behavior of fork to replicate process memory and transmit file descriptors. This PR refactors the Spack internals to support starting a child process with the "spawn" method. To achieve this, it makes the following changes: - ensure that the package repository and other global state are transmitted to the child process - ensure that file descriptors are transmitted to the child process in a way that works with multiprocessing and spawn - make all the state needed for the build process and tests picklable (package, stage, etc.) - move a number of locally-defined functions into global scope so that they can be pickled - rework tests where needed to avoid using local functions This PR also reworks sbang tests to work on macOS, where temporary directories are deeper than the Linux sbang limit. We make the limit platform-dependent (macOS supports 512-character shebangs) See: #14102
2020-10-23csh: don't require SPACK_ROOT for sourcing setup-env.csh (#18225)Todd Gamblin2-5/+10
Don't require SPACK_ROOT for sourcing setup-env.csh and make output more consistent
2020-10-11Add testing for Python 3.9 (#19261)Adam J. Stewart5-11/+11
2020-10-11Fix failing mpich build tests (#19259)Adam J. Stewart1-1/+1
By default Spack uses the latest (highest version) GCC compiler available, which might change across updates of the Github CI environment. Since a C compiler is always installed and `mpich~fortran` will result in faster build times, avoid building the FORTRAN interface as part of the test.
2020-09-28macOS CI: replace jupyter with jupyterlab (#19029)Adam J. Stewart1-2/+1
2020-09-02Add new RubyPackage build system base class (#18199)Adam J. Stewart1-2/+11
* Add new RubyPackage build system base class * Ruby: add spack external find support * Add build tests for RubyPackage
2020-09-02Mac OS: support Python >= 3.8 by using fork-based multiprocessing (#18124)Rui Xue3-6/+6
As detailed in https://bugs.python.org/issue33725, starting new processes with 'fork' on Mac OS is not guaranteed to work in general. As of Python 3.8 the default process spawning mechanism was changed to avoid this issue. Spack depends on the fork-based method to preserve file descriptors transparently, to preserve global state, and to avoid pickling some objects. An effort is underway to remove dependence on fork-based process spawning (see #18205). In the meantime, this allows Spack to run with Python 3.8 on Mac OS by explicitly choosing to use 'fork'. Co-authored-by: Peter Josef Scheibel <scheibel1@llnl.gov> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2020-08-03MacOS nightly builds: use Python 3.7 in CIMassimiliano Culpo1-0/+12
Nightly builds with MacOS started failing again due to an upgrade of the default virtual environment that now uses Python 3.8 This makes us hit #14102 and every build fails. This commit should be reverted along with the fix to #14102.
2020-08-01Avoid update and upgrades to brew (#17815)Massimiliano Culpo1-2/+0
Ci is currently failing on brew update with the error: ``` Error: Cannot install bazelisk because conflicting formulae are installed. bazel: because Bazelisk replaces the bazel binary Please `brew unlink bazel` before continuing. Unlinking removes a formula's symlinks from /usr/local. You can link the formula again after the install finishes. You can --force this install, but the build may fail or cause obscure side effects in the resulting software. ``` Avoiding: ``` $ brew update $ brew upgrade ``` solves the issue by preventing the risk of conflicting formulae
2020-07-31Move Python 2.6 unit tests to Github Actions (#17279)Massimiliano Culpo1-2/+15
* Run Python2.6 unit tests on Github Actions * Skip url tests on Python 2.6 to reduce waiting times * Skip foreground background tests on Python 2.6 to reduce waiting times * Removed references to Travis in the documentation * Deleted install_patchelf.sh (can be installed from repo on CentOS 6)
2020-07-29Use "fetch-depth: 0" to retrieve all history from remoteMassimiliano Culpo4-10/+19
2020-07-29Simplified YAML files for Github Actions workflowsMassimiliano Culpo4-18/+7
Updated actions where needed
2020-07-29Group tests with similar duration togetherMassimiliano Culpo4-76/+72
Style and documentation tests take just a few minutes to run. Since in Github actions one can't restart a single job but needs to restart an entire workflow, here we group tests with similar duration together.