summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2020-12-22Refactor flake8 handling and tool compatibility (#20376)Tom Scogland5-254/+292
This PR does three related things to try to improve developer tooling quality of life: 1. Adds new options to `.flake8` so it applies the rules of both `.flake8` and `.flake_package` based on paths in the repository. 2. Adds a re-factoring of the `spack flake8` logic into a flake8 plugin so using flake8 directly, or through editor or language server integration, only reports errors that `spack flake8` would. 3. Allows star import of `spack.pkgkit` in packages, since this is now the thing that needs to be imported for completion to work correctly in package files, it's nice to be able to do that. I'm sorely tempted to sed over the whole repository and put `from spack.pkgkit import *` in every package, but at least being allowed to do it on a per-package basis helps. As an example of what the result of this is: ``` ~/Workspace/Projects/spack/spack develop* ⇣ ❯ flake8 --format=pylint ./var/spack/repos/builtin/packages/kripke/package.py ./var/spack/repos/builtin/packages/kripke/package.py:6: [F403] 'from spack.pkgkit import *' used; unable to detect undefined names ./var/spack/repos/builtin/packages/kripke/package.py:25: [E501] line too long (88 > 79 characters) ~/Workspace/Projects/spack/spack refactor-flake8* 1 ❯ flake8 --format=spack ./var/spack/repos/builtin/packages/kripke/package.py ~/Workspace/Projects/spack/spack refactor-flake8* ❯ flake8 ./var/spack/repos/builtin/packages/kripke/package.py ``` * qa/flake8: update .flake8, spack formatter plugin Adds: * Modern flake8 settings for per-path/glob error ignores, allows packages to use the same `.flake8` as the rest of spack * A spack formatter plugin to flake8 that implements the behavior of `spack flake8` for direct invocations. Makes integration with developer tooling nicer, linting with flake8 reports only errors that `spack flake8` would report. Using pyls and pyls-flake8, or any other non-format-dependent flake8 integration, now works with spack's rules. * qa/flake8: allow star import of spack.pkgkit To get working completion of directives and spack components it's necessary to import the contents of spack.pkgkit. At the moment doing this makes flake8 displeased. For now, allow spack.pkgkit and spack both, next step is to ban spack * and require spack.pkgkit *. * first cut at refactoring spack flake8 This version still copies all of the files to be checked as befire, and some other things that probably aren't necessary, but it relies on the spack formatter plugin to implement the ignore logic. * keep flake8 from rejecting itself * remove separate packages flake8 config * fix failures from too many files I ran into this in the PR converting pkgkit to std. The solution in that branch does not work in all cases as it turns out, and all the workarounds I tried to use generated configs to get a single invocation of flake8 with a filename optoion to work failed. It's an astonishingly frustrating config option. Regardless, this removes all temporary file creation from the command and relies on the plugin instead. To work around the huge number of files in spack and still allow the command to control what gets checked, it scans files in batches of 100. This is a completely arbitrary number but was chosen to be safely under common line-length limits. One side-effect of this is that every 100 files the command will produce output, rather than only at the end, which doesn't seem like a terrible thing.
2020-12-22concretizer: optimize loop on compiler versionMassimiliano Culpo2-13/+15
Similar to the optimization on platform
2020-12-22concretizer: optimized loop on node platformsMassimiliano Culpo1-3/+3
We can speed-up the computation by avoiding a double loop in a cardinality constraint and enforcing the rule instead as an integrity constraint.
2020-12-21Corrected Go dependent build environment. (#18493)Paul1-7/+8
* Dependencies of Go will now correctly set the GOPATH for the appropriate spec to avoid using the user's default path. * Bumped version to latest releases(1.15.6 & 1.14.13).
2020-12-21ParaView: add new ParaView-5.9.0-RC3 release (#20500)Vicente Bolea1-1/+1
Signed-off-by: Vicente Adolfo Bolea Sanchez <vicente.bolea@kitware.com>
2020-12-21clingo: prefer master branchTodd Gamblin1-1/+1
Most people installing `clingo` with Spack are going to be doing it to use the new concretizer, and that requires the `master` branch. - [x] make `master` the default so we don't have to keep telling people to install `clingo@master`. We'll update the preferred version when there's a new release.
2020-12-21New package: PMERGE (#20390)downloadico1-0/+29
2020-12-21upstream dependencies changes (#20493)Karen C. Tsai1-13/+19
* update dependencies and cmake options * add blank line
2020-12-21Added support for AOCC compileramd-toolchain-support1-2/+4
2020-12-20geant4-vmc: Add version 5-3, switch to https (#20379)Dr. Christian Tacke1-3/+4
2020-12-20concretizer: fix failing unit testsMassimiliano Culpo2-4/+18
2020-12-20concretizer: emit facts for integrity constraintsMassimiliano Culpo2-37/+40
2020-12-20concretizer: emit facts for constraints on imposed dependenciesMassimiliano Culpo2-38/+85
2020-12-20concretizer: avoid redundant grounding on dependency typesMassimiliano Culpo2-27/+24
2020-12-20concretizer: move conditional dependency logic into `concretize.lp`Todd Gamblin2-18/+60
Continuing to convert everything in `asp.py` into facts, make the generation of ground rules for conditional dependencies use facts, and move the semantics into `concretize.lp`. This is probably the most complex logic in Spack, as dependencies can be conditional on anything, and we need conditional ASP rules to accumulate and map all the dependency conditions to spec attributes. The logic looks complicated, but essentially it accumulates any constraints associated with particular conditions into a fact associated with the condition by id. Then, if *any* condition id's fact is True, we trigger the dependency. This simplifies the way `declared_dependency()` works -- the dependency is now declared regardless of whether it is conditional, and the conditions are handled by `dependency_condition()` facts.
2020-12-20concretizer: spec_clauses should traverse dependenciesTodd Gamblin1-20/+24
There are currently no places where we do not want to traverse dependencies in `spec_clauses()`, so simplify the logic by consolidating `spec_traverse_clauses()` with `spec_clauses()`.
2020-12-20concretizer: pull _develop_specs_from_env out of main setup loopTodd Gamblin1-7/+10
2020-12-20concretizer: add #defined statements to avoid warnings.Todd Gamblin1-0/+2
`version_satisfies/2` and `node_compiler_version_satisfies/3` are generated but need `#defined` directives to avoid " info: atom does not occur in any rule head:" warnings.
2020-12-19asp: memoize the list of all target_specs to speed-up setup phase (#20473)Massimiliano Culpo1-4/+11
* asp: memoize the list of all target_specs to speed-up setup phase * asp: memoize using a cache per solver object
2020-12-18minimal zsh completion (#20253)Tom Scogland5-35/+74
Since zsh can load bash completion files natively, seems reasonable to just turn this on. The only changes are to switch from `type -t` which zsh doesn't support to using `type` with a regex and adding a new arm to the sourcing of the completions to allow it to work for zsh as well as bash. Could use more bash/dash/etc testing probably, but everything I've thought to try has worked so far. Notes: * unit-test zsh support, fix issues Specifically fixed word splitting in completion-test, use a different method to apply sh emulation to zsh loaded bash completion, and fixed an incompatibility in regex operator quoting requirements. * compinit now ignores insecure directories Completion isn't meant to be enabled in non-interactive environments, so by default compinit will ask the user if they want to ignore insecure directories or load them anyway. To pass the spack unit tests in GH actions, this prompt must be disabled, so ignore explicitly until a better solution can be found. * debug functions test also requires bash emulation COMP_WORDS is a bash-ism that zsh doesn't natively support, turn on emulation for just that section of tests to allow the comparison to work. Does not change the behavior of the functions themselves since they are already pinned to sh emulation elsewhere. * propagate change to .in file * fix comment and update script based on .in
2020-12-18Add spack test support for Qthreads (#20437)Jan Ciesko1-1/+52
2020-12-18slepc: set --with-arpack-dir correctly (#20463)eugeneswalker1-1/+1
2020-12-18mesa: drop use of shared variant (#20453)Chuck Atkins1-4/+0
Mesa no longer supports building static libraries so it doesn't even make sense to add the variant.
2020-12-18ci: fixes for compiler bootstrapping (#17563)Scott Wittenburg7-63/+241
This PR addresses a number of issues related to compiler bootstrapping. Specifically: 1. Collect compilers to be bootstrapped while queueing in installer Compiler tasks currently have an incomplete list in their task.dependents, making those packages fail to install as they think they have not all their dependencies installed. This PR collects the dependents and sets them on compiler tasks. 2. allow boostrapped compilers to back off target Bootstrapped compilers may be built with a compiler that doesn't support the target used by the rest of the spec. Allow them to build with less aggressive target optimization settings. 3. Support for target ranges Backing off the target necessitates computing target ranges, so make Spack handle those properly. Notably, this adds an intersection method for target ranges and fixes the way ranges are satisfied and constrained on Spec objects. This PR also: - adds testing - improves concretizer handling of target ranges Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com> Co-authored-by: Gregory Becker <becker33@llnl.gov> Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2020-12-17py-greenlet: add version 0.4.17 (#20457)eugeneswalker1-1/+4
* py-greenlet: add version 0.4.17 * py-greenlet depends on python for build, link, and run
2020-12-17meson: Add cmake runtime dependency (#20449)Chuck Atkins1-0/+1
2020-12-17New package: cray-mpich (#20076)Harmen Stoppels2-27/+69
Cray's version of MPICH uses a different versioning system than MPICH, so it has been split into its own package. It is an external-only package (always provided by the system, never installed by Spack).
2020-12-17superlu depends on tcsh for build (#20381)eugeneswalker1-0/+1
2020-12-17cmake: Add Version 3.19.2 (#20448)Dr. Christian Tacke1-0/+1
2020-12-17View location resolve environment variables (#20420)psakievich1-1/+2
2020-12-17add py-gemini (#16950)manifest2-0/+88
* py-gemini + * dep py-setuptools + * deps grabix+ * py-gemini dep version fix * grabix compiler fix * Update package.py Typo correction
2020-12-17biobambam2: new package at v2.0.177 (#20392)takanori-ihara1-0/+26
2020-12-17pangolin: add new package (#20388)Adam J. Stewart1-0/+22
2020-12-17kim-api: added v2.2.1 (#20382)Ryan S. Elliott1-1/+2
2020-12-17suite-sparse: add flags for Fujitsu compilers (#20393)ketsubouchi1-2/+2
2020-12-17docs: add single node concurrent build example (#20416)Tamara Dahlgren1-4/+11
2020-12-17OpenCV: add versions up to v4.5.0, variants (#20378)Adam J. Stewart1-29/+27
2020-12-17flexi: new package (#20313)Michael Kuhn1-0/+41
2020-12-17openblas: added v0.3.13 (#20357)Michael Kuhn1-0/+6
2020-12-17mbedtls: added v2.16.9 (#20422)Ryan Mast1-0/+1
2020-12-17DBoW2: add new package (#20387)Adam J. Stewart1-0/+21
2020-12-17acts: added v3.00.0 (#20402)Hadrien G1-0/+1
2020-12-17arm: added new package (#18019)Massimiliano Culpo1-0/+86
The package is at the moment not installable, just detectable.
2020-12-17xlc, xlf: added new packages (#18154)Massimiliano Culpo2-0/+150
2020-12-17unit-tests: ensure that installed packages can be reused (#20307)Massimiliano Culpo1-0/+84
refers #20292 Added a unit test that ensures we can reuse installed packages even if in the repository variants have been removed or added.
2020-12-17help2man: update to 1.47.16 (#20397)Ryan Mast1-0/+1
2020-12-16sundials: add versions up to 5.6.1 (#20436)Ryan Mast1-0/+2
2020-12-16Added PyAMG package. (#20414)Ben Cowan1-0/+25
2020-12-16cli11: add new package (#20398)Ryan Mast1-0/+26
2020-12-16PythonPackage: add import module smoke tests (#20023)Adam J. Stewart241-1046/+335