summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2020-12-23concretizer: remove vestigial code and commentMassimiliano Culpo1-20/+0
2020-12-23style: ensure that all packages pass `spack style -a`Todd Gamblin31-114/+122
- fix trailing whitespace and other issues uncovered by better flake8 checking. - fix extra whitespace printed by `spack style` command
2020-12-23Add Intel oneAPI packages (#20411)Robert Cohn14-20/+398
This creates a set of packages which all use the same script to install components of Intel oneAPI. This includes: * An inheritable IntelOneApiPackage which knows how to invoke the installation script based on which components are requested * For components which include headers/libraries, an inheritable IntelOneApiLibraryPackage is provided to locate them * Individual packages for DAL, DNN, TBB, etc. * A package for the Intel oneAPI compilers (icx/ifx). This also includes icc/ifortran but these are not currently detected in this PR
2020-12-23papi: patch fortran tests for Cray FTN compiler (#20339)Howard Pritchard2-0/+58
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
2020-12-22add mypy to style checks; rename `spack flake8` to `spack style` (#20384)Tom Scogland50-317/+661
I lost my mind a bit after getting the completion stuff working and decided to get Mypy working for spack as well. This adds a `.mypy.ini` that checks all of the spack and llnl modules, though not yet packages, and fixes all of the identified missing types and type issues for the spack library. In addition to these changes, this includes: * rename `spack flake8` to `spack style` Aliases flake8 to style, and just runs flake8 as before, but with a warning. The style command runs both `flake8` and `mypy`, in sequence. Added --no-<tool> options to turn off one or the other, they are on by default. Fixed two issues caught by the tools. * stub typing module for python2.x We don't support typing in Spack for python 2.x. To allow 2.x to support `import typing` and `from typing import ...` without a try/except dance to support old versions, this adds a stub module *just* for python 2.x. Doing it this way means we can only reliably use all type hints in python3.7+, and mypi.ini has been updated to reflect that. * add non-default black check to spack style This is a first step to requiring black. It doesn't enforce it by default, but it will check it if requested. Currently enforcing the line length of 79 since that's what flake8 requires, but it's a bit odd for a black formatted project to be quite that narrow. All settings are in the style command since spack has no pyproject.toml and I don't want to add one until more discussion happens. Also re-format `style.py` since it no longer passed the black style check with the new length. * use style check in github action Update the style and docs action to use `spack style`, adding in mypy and black to the action even if it isn't running black right now.
2020-12-22Add patch to fix bazel build on power9 (#20512)Harsh Bhatia2-0/+71
* fix bazel on power9 * small fix in tensorflow package * removed import not needed anymore
2020-12-22concretizer: refactor conditional rules to be less repetitious (#20507)Todd Gamblin2-89/+60
We have to repeat all the spec attributes in a number of places in `concretize.lp`, and Spack has a fair number of spec attributes. If we instead add some rules up front that establish equivalencies like this: ``` node(Package) :- attr("node", Package). attr("node", Package) :- node(Package). version(Package, Version) :- attr("version", Package, Version). attr("version", Package, Version) :- version(Package, Version). ``` We can rewrite most of the repetitive conditions with `attr` and repeat only for each arity (there are only 3 arities for spec attributes so far) as opposed to each spec attribute. This makes the logic easier to read and the rules easier to follow. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2020-12-22Patch meson.build llvm config strategy (#20451)Ben Bergen1-0/+6
This patch logic resovles a linking issue with ncurses in the mesa package. This appears to be a recurring problem that was identified in the mesa gitlab issues here: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2843 Using `_llvm_method = 'auto'` is broken. This patch replaces that with `_llvm_method = 'config-tool'`, which is a hack, but makes it possible to build. I have commented on the closed issue (2843), referencing the original author of the bug, and one of the mesa developers, so perhaps they will fix the problem.
2020-12-22Refactor flake8 handling and tool compatibility (#20376)Tom Scogland5-254/+292
This PR does three related things to try to improve developer tooling quality of life: 1. Adds new options to `.flake8` so it applies the rules of both `.flake8` and `.flake_package` based on paths in the repository. 2. Adds a re-factoring of the `spack flake8` logic into a flake8 plugin so using flake8 directly, or through editor or language server integration, only reports errors that `spack flake8` would. 3. Allows star import of `spack.pkgkit` in packages, since this is now the thing that needs to be imported for completion to work correctly in package files, it's nice to be able to do that. I'm sorely tempted to sed over the whole repository and put `from spack.pkgkit import *` in every package, but at least being allowed to do it on a per-package basis helps. As an example of what the result of this is: ``` ~/Workspace/Projects/spack/spack develop* ⇣ ❯ flake8 --format=pylint ./var/spack/repos/builtin/packages/kripke/package.py ./var/spack/repos/builtin/packages/kripke/package.py:6: [F403] 'from spack.pkgkit import *' used; unable to detect undefined names ./var/spack/repos/builtin/packages/kripke/package.py:25: [E501] line too long (88 > 79 characters) ~/Workspace/Projects/spack/spack refactor-flake8* 1 ❯ flake8 --format=spack ./var/spack/repos/builtin/packages/kripke/package.py ~/Workspace/Projects/spack/spack refactor-flake8* ❯ flake8 ./var/spack/repos/builtin/packages/kripke/package.py ``` * qa/flake8: update .flake8, spack formatter plugin Adds: * Modern flake8 settings for per-path/glob error ignores, allows packages to use the same `.flake8` as the rest of spack * A spack formatter plugin to flake8 that implements the behavior of `spack flake8` for direct invocations. Makes integration with developer tooling nicer, linting with flake8 reports only errors that `spack flake8` would report. Using pyls and pyls-flake8, or any other non-format-dependent flake8 integration, now works with spack's rules. * qa/flake8: allow star import of spack.pkgkit To get working completion of directives and spack components it's necessary to import the contents of spack.pkgkit. At the moment doing this makes flake8 displeased. For now, allow spack.pkgkit and spack both, next step is to ban spack * and require spack.pkgkit *. * first cut at refactoring spack flake8 This version still copies all of the files to be checked as befire, and some other things that probably aren't necessary, but it relies on the spack formatter plugin to implement the ignore logic. * keep flake8 from rejecting itself * remove separate packages flake8 config * fix failures from too many files I ran into this in the PR converting pkgkit to std. The solution in that branch does not work in all cases as it turns out, and all the workarounds I tried to use generated configs to get a single invocation of flake8 with a filename optoion to work failed. It's an astonishingly frustrating config option. Regardless, this removes all temporary file creation from the command and relies on the plugin instead. To work around the huge number of files in spack and still allow the command to control what gets checked, it scans files in batches of 100. This is a completely arbitrary number but was chosen to be safely under common line-length limits. One side-effect of this is that every 100 files the command will produce output, rather than only at the end, which doesn't seem like a terrible thing.
2020-12-22concretizer: optimize loop on compiler versionMassimiliano Culpo2-13/+15
Similar to the optimization on platform
2020-12-22concretizer: optimized loop on node platformsMassimiliano Culpo1-3/+3
We can speed-up the computation by avoiding a double loop in a cardinality constraint and enforcing the rule instead as an integrity constraint.
2020-12-21Corrected Go dependent build environment. (#18493)Paul1-7/+8
* Dependencies of Go will now correctly set the GOPATH for the appropriate spec to avoid using the user's default path. * Bumped version to latest releases(1.15.6 & 1.14.13).
2020-12-21ParaView: add new ParaView-5.9.0-RC3 release (#20500)Vicente Bolea1-1/+1
Signed-off-by: Vicente Adolfo Bolea Sanchez <vicente.bolea@kitware.com>
2020-12-21clingo: prefer master branchTodd Gamblin1-1/+1
Most people installing `clingo` with Spack are going to be doing it to use the new concretizer, and that requires the `master` branch. - [x] make `master` the default so we don't have to keep telling people to install `clingo@master`. We'll update the preferred version when there's a new release.
2020-12-21New package: PMERGE (#20390)downloadico1-0/+29
2020-12-21upstream dependencies changes (#20493)Karen C. Tsai1-13/+19
* update dependencies and cmake options * add blank line
2020-12-21Added support for AOCC compileramd-toolchain-support1-2/+4
2020-12-20geant4-vmc: Add version 5-3, switch to https (#20379)Dr. Christian Tacke1-3/+4
2020-12-20concretizer: fix failing unit testsMassimiliano Culpo2-4/+18
2020-12-20concretizer: emit facts for integrity constraintsMassimiliano Culpo2-37/+40
2020-12-20concretizer: emit facts for constraints on imposed dependenciesMassimiliano Culpo2-38/+85
2020-12-20concretizer: avoid redundant grounding on dependency typesMassimiliano Culpo2-27/+24
2020-12-20concretizer: move conditional dependency logic into `concretize.lp`Todd Gamblin2-18/+60
Continuing to convert everything in `asp.py` into facts, make the generation of ground rules for conditional dependencies use facts, and move the semantics into `concretize.lp`. This is probably the most complex logic in Spack, as dependencies can be conditional on anything, and we need conditional ASP rules to accumulate and map all the dependency conditions to spec attributes. The logic looks complicated, but essentially it accumulates any constraints associated with particular conditions into a fact associated with the condition by id. Then, if *any* condition id's fact is True, we trigger the dependency. This simplifies the way `declared_dependency()` works -- the dependency is now declared regardless of whether it is conditional, and the conditions are handled by `dependency_condition()` facts.
2020-12-20concretizer: spec_clauses should traverse dependenciesTodd Gamblin1-20/+24
There are currently no places where we do not want to traverse dependencies in `spec_clauses()`, so simplify the logic by consolidating `spec_traverse_clauses()` with `spec_clauses()`.
2020-12-20concretizer: pull _develop_specs_from_env out of main setup loopTodd Gamblin1-7/+10
2020-12-20concretizer: add #defined statements to avoid warnings.Todd Gamblin1-0/+2
`version_satisfies/2` and `node_compiler_version_satisfies/3` are generated but need `#defined` directives to avoid " info: atom does not occur in any rule head:" warnings.
2020-12-19asp: memoize the list of all target_specs to speed-up setup phase (#20473)Massimiliano Culpo1-4/+11
* asp: memoize the list of all target_specs to speed-up setup phase * asp: memoize using a cache per solver object
2020-12-18minimal zsh completion (#20253)Tom Scogland5-35/+74
Since zsh can load bash completion files natively, seems reasonable to just turn this on. The only changes are to switch from `type -t` which zsh doesn't support to using `type` with a regex and adding a new arm to the sourcing of the completions to allow it to work for zsh as well as bash. Could use more bash/dash/etc testing probably, but everything I've thought to try has worked so far. Notes: * unit-test zsh support, fix issues Specifically fixed word splitting in completion-test, use a different method to apply sh emulation to zsh loaded bash completion, and fixed an incompatibility in regex operator quoting requirements. * compinit now ignores insecure directories Completion isn't meant to be enabled in non-interactive environments, so by default compinit will ask the user if they want to ignore insecure directories or load them anyway. To pass the spack unit tests in GH actions, this prompt must be disabled, so ignore explicitly until a better solution can be found. * debug functions test also requires bash emulation COMP_WORDS is a bash-ism that zsh doesn't natively support, turn on emulation for just that section of tests to allow the comparison to work. Does not change the behavior of the functions themselves since they are already pinned to sh emulation elsewhere. * propagate change to .in file * fix comment and update script based on .in
2020-12-18Add spack test support for Qthreads (#20437)Jan Ciesko1-1/+52
2020-12-18slepc: set --with-arpack-dir correctly (#20463)eugeneswalker1-1/+1
2020-12-18mesa: drop use of shared variant (#20453)Chuck Atkins1-4/+0
Mesa no longer supports building static libraries so it doesn't even make sense to add the variant.
2020-12-18ci: fixes for compiler bootstrapping (#17563)Scott Wittenburg7-63/+241
This PR addresses a number of issues related to compiler bootstrapping. Specifically: 1. Collect compilers to be bootstrapped while queueing in installer Compiler tasks currently have an incomplete list in their task.dependents, making those packages fail to install as they think they have not all their dependencies installed. This PR collects the dependents and sets them on compiler tasks. 2. allow boostrapped compilers to back off target Bootstrapped compilers may be built with a compiler that doesn't support the target used by the rest of the spec. Allow them to build with less aggressive target optimization settings. 3. Support for target ranges Backing off the target necessitates computing target ranges, so make Spack handle those properly. Notably, this adds an intersection method for target ranges and fixes the way ranges are satisfied and constrained on Spec objects. This PR also: - adds testing - improves concretizer handling of target ranges Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com> Co-authored-by: Gregory Becker <becker33@llnl.gov> Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2020-12-17py-greenlet: add version 0.4.17 (#20457)eugeneswalker1-1/+4
* py-greenlet: add version 0.4.17 * py-greenlet depends on python for build, link, and run
2020-12-17meson: Add cmake runtime dependency (#20449)Chuck Atkins1-0/+1
2020-12-17New package: cray-mpich (#20076)Harmen Stoppels2-27/+69
Cray's version of MPICH uses a different versioning system than MPICH, so it has been split into its own package. It is an external-only package (always provided by the system, never installed by Spack).
2020-12-17superlu depends on tcsh for build (#20381)eugeneswalker1-0/+1
2020-12-17cmake: Add Version 3.19.2 (#20448)Dr. Christian Tacke1-0/+1
2020-12-17View location resolve environment variables (#20420)psakievich1-1/+2
2020-12-17add py-gemini (#16950)manifest2-0/+88
* py-gemini + * dep py-setuptools + * deps grabix+ * py-gemini dep version fix * grabix compiler fix * Update package.py Typo correction
2020-12-17biobambam2: new package at v2.0.177 (#20392)takanori-ihara1-0/+26
2020-12-17pangolin: add new package (#20388)Adam J. Stewart1-0/+22
2020-12-17kim-api: added v2.2.1 (#20382)Ryan S. Elliott1-1/+2
2020-12-17suite-sparse: add flags for Fujitsu compilers (#20393)ketsubouchi1-2/+2
2020-12-17docs: add single node concurrent build example (#20416)Tamara Dahlgren1-4/+11
2020-12-17OpenCV: add versions up to v4.5.0, variants (#20378)Adam J. Stewart1-29/+27
2020-12-17flexi: new package (#20313)Michael Kuhn1-0/+41
2020-12-17openblas: added v0.3.13 (#20357)Michael Kuhn1-0/+6
2020-12-17mbedtls: added v2.16.9 (#20422)Ryan Mast1-0/+1
2020-12-17DBoW2: add new package (#20387)Adam J. Stewart1-0/+21
2020-12-17acts: added v3.00.0 (#20402)Hadrien G1-0/+1