summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-02-17Tests: enable re-use of post-install tests in smoke tests (#20298)Tamara Dahlgren2-3/+8
2021-02-17concretizer: try hard to obtain all needed variant_possible_value()'s (#20102)Andrew W Elble7-10/+87
Track all the variant values mentioned when emitting constraints, validate them and emit a fact that allows them as possible values. This modification ensures that open-ended variants (variants accepting any string or any integer) are projected to the finite set of values that are relevant for this concretization.
2021-02-17bugfix: work around issue handling packages not in any repoMassimiliano Culpo2-0/+7
2021-02-17concretizer: refactor handling of special variants dev_build and patchesTodd Gamblin2-50/+55
Other parts of the concretizer code build up lists of things we can't know without traversing all specs and packages, and they output these list at the very end. The code for this for variant values from spec literals was intertwined with the code for traversing the input specs. This only covers the input specs and misses variant values that might come from directives in packages. - [x] move ad-hoc value handling code into spec_clauses so we do it in one place for CLI and packages - [x] move handling of `variant_possible_value`, etc. into `concretize.lp`, where we can automatically infer variant existence more concisely. - [x] simplify/clarify some of the code for variants in `spec_clauses()`
2021-02-17VTK-m: update to specify correct requirements to kokkos (#20097)Robert Maynard1-5/+13
2021-02-17concretizer: each external version is allowed by definition (#20247)Massimiliano Culpo5-0/+42
Registering external versions among the lists of allowed ones generates the correct rules for `version_satisfies`
2021-02-17concretizer: restrict maximizing variant values to MV variants (#20194)Massimiliano Culpo4-2/+33
2021-02-17environment installs: fix reporting. (#20004)Matthias Wolf1-1/+1
PR #15702 changed the invocation of the report context when installing specs, do the same when building environments.
2021-02-17avoid circular import (#20236)Greg Becker1-1/+1
2021-02-17concretizer: call inject_patches_variants() on the roots of the specs (#20203)Andrew W Elble5-4/+31
As was done in the old concretizer. Fixes an issue where conditionally patched dependencies did not show up in spec (gdal+jasper)
2021-02-17concretizer: try hard to infer the real version of compilers (#20099)Massimiliano Culpo3-3/+53
fixes #20055 Compiler with custom versions like gcc@foo are not currently matched to the appropriate targets. This is because the version of spec doesn't match the "real" version of the compiler. This PR replicates the strategy used in the original concretizer to deal with that and tries to detect the real version of compilers if the version in the spec returns no results.
2021-02-17Fix hipcc once more (#20095)Harmen Stoppels8-90/+90
2021-02-17boost: disable find_package's config mode for boost prior to v1.70.0 (#20198)Massimiliano Culpo1-0/+13
2021-02-17concretizer: don't optimize emitting version_satisfies() (#20128)Andrew W Elble2-4/+5
When all versions were allowed a version_satisfies rule was not emitted, and this caused conditional directives to fail.
2021-02-17Fixes compile time errors (#20006)MichaelLaufer1-2/+4
Co-authored-by: michael laufer <michael.laufer@toganetworks.com>
2021-02-17spec: return early from concretization if a spec is already concrete (#20196)Massimiliano Culpo1-0/+3
2021-02-17concretizer: remove ad-hoc rule for external packages (#20193)Massimiliano Culpo4-7/+42
fixes #20040 Matching compilers among nodes has been prioritized in #20020. Selection of default variants has been tuned in #20182. With this setup there is no need to have an ad-hoc rule for external packages. On the contrary it should be removed to prefer having default variant values over more external nodes in the DAG.
2021-02-17concretizer: swap priority of selecting provider and default variant (#20182)Massimiliano Culpo5-13/+72
refers #20040 Before this PR optimization rules would have selected default providers at a higher priority than default variants. Here we swap this priority and we consider variants that are forced by any means (root spec or spec in depends_on clause) the same as if they were with a default value. This prevents the solver from avoiding expected configurations just because they contain directives like: depends_on('pkg+foo') and `+foo` is not the default variant value for pkg.
2021-02-17Typos: add missing closing parens (#20174)George Hartzell1-3/+3
2021-02-17concretizer: treat target ranges in directives correctly (#19988)Massimiliano Culpo3-1/+74
fixes #19981 This commit adds support for target ranges in directives, for instance: conflicts('+foo', when='target=x86_64:,aarch64:') If any target in a spec body is not a known target the following clause will be emitted: node_target_satisfies(Package, TargetConstraint) when traversing the spec and a definition of the clause will then be printed at the end similarly to what is done for package and compiler versions.
2021-02-17concretizer: prioritize matching compilers over newer versions (#20020)Massimiliano Culpo4-6/+37
fixes #20019 Before this modification having a newer version of a node came at higher priority in the optimization than having matching compilers. This could result in unexpected configurations for packages with conflict directives on compilers of the type: conflicts('%gcc@X.Y:', when='@:A.B') where changing the compiler for just that node is preferred to lower the node version to less than 'A.B'. Now the priority has been switched so the solver will try to lower the version of the nodes in question before changing their compiler.
2021-02-17concretizer: allow a bool to be passed as argument for tests dependencies ↵Massimiliano Culpo6-8/+69
(#20082) refers #20079 Added docstrings to 'concretize' and 'concretized' to document the format for tests. Added tests for the activation of test dependencies.
2021-02-17concretizer: treat conditional providers correctly (#20086)Massimiliano Culpo4-1/+49
refers #20040 This modification emits rules like: provides_virtual("netlib-lapack","blas") :- variant_value("netlib-lapack","external-blas","False"). for packages that provide virtual dependencies conditionally instead of a fact that doesn't account for the condition.
2021-02-17Docs: remove duplication in Command Reference (#20021)Adam J. Stewart2-2/+2
2021-02-17recognize macOS 11.1 as big sur (#20038)Martin Aumüller1-2/+6
Big Sur versions go 11.0, 11.0.1, 11.1 (vs. prior versions that only used the minor component) Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2021-02-17py-ipykernel: fix bug in phase method (#19986)Adam J. Stewart1-2/+3
* py-ipykernel: fix bug in phase method * Fix bug in executable calling
2021-02-17fix error handling for spack test results command (#19987)Greg Becker1-0/+1
2021-02-17hip support for umpire, chai, raja, camp (#19715)Danny Taller7-43/+232
* create HipPackage base class and do some refactoring * comments and added conflict to raja for openmp with hip
2021-02-17py-ipykernel: fix install (#19617)Axel Huebl1-0/+6
There is a post-install routine in `ipykernel` that needs to be called for proper registration with jupyter.
2020-11-18update CHANGELOG.md for v0.16.0v0.16.0Todd Gamblin1-0/+110
2020-11-18bump version number to 0.16.0Todd Gamblin1-1/+1
2020-11-18clingo: add `master` branch version (#19958)Massimiliano Culpo1-1/+3
* updated @master to point to the master branch * also added a @spack that points to a fixed commit
2020-11-18cmd: add `spack mark` command (#16662)Michael Kuhn8-34/+339
This adds a new `mark` command that can be used to mark packages as either explicitly or implicitly installed. Apart from fixing the package database after installing a dependency manually, it can be used to implement upgrade workflows as outlined in #13385. The following commands demonstrate how the `mark` and `gc` commands can be used to only keep the current version of a package installed: ```console $ spack install pkgA $ spack install pkgB $ git pull # Imagine new versions for pkgA and/or pkgB are introduced $ spack mark -i -a $ spack install pkgA $ spack install pkgB $ spack gc ``` If there is no new version for a package, `install` will simply mark it as explicitly installed and `gc` will not remove it. Co-authored-by: Greg Becker <becker33@llnl.gov>
2020-11-18spack test (#15702)Greg Becker131-676/+3599
Users can add test() methods to their packages to run smoke tests on installations with the new `spack test` command (the old `spack test` is now `spack unit-test`). spack test is environment-aware, so you can `spack install` an environment and then run `spack test run` to run smoke tests on all of its packages. Historical test logs can be perused with `spack test results`. Generic smoke tests for MPI implementations, C, C++, and Fortran compilers as well as specific smoke tests for 18 packages. Inside the test method, individual tests can be run separately (and continue to run best-effort after a test failure) using the `run_test` method. The `run_test` method encapsulates finding test executables, running and checking return codes, checking output, and error handling. This handles the following trickier aspects of testing with direct support in Spack's package API: - [x] Caching source or intermediate build files at build time for use at test time. - [x] Test dependencies, - [x] packages that require a compiler for testing (such as library only packages). See the packaging guide for more details on using Spack testing support. Included is support for package.py files for virtual packages. This does not change the Spack interface, but is a major change in internals. Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov> Co-authored-by: wspear <wjspear@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-11-17Added -level_zero -rocm -opencl flags and sha256 for TAU v2.30. (#19962)sameershende1-1/+18
* Added -level_zero -rocm -opencl flags and sha256 for TAU v2.30. * Removed the depends_on clause for OpenCL and added a variant for OneAPI level_zero. * remove depends_on rocm * remove depends_on rocprofiler Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
2020-11-17Improve warning message for deprecated attributes in "packages.yaml"Massimiliano Culpo6-28/+67
The deprecatedProperties custom validator now can accept a function to compute a better error message. Improve error/warning message for deprecated properties
2020-11-17Documentation: spack load/environments prefix inspections (#19961)Peter Scheibel3-4/+34
As of #18260, `spack load` and `spack env activate` now use `prefix_inspections` from the modules configuration to decide how to modify environment variables. This updates the modules configuration documentation to describe how to update environment variables with the `prefix_inspections` section. This also updates the `spack load` and environments documentation to refer to the new `prefix_inspections` documentation.
2020-11-17spack load/environments: allow customization of prefix inspections (#18260)Dr. Christian Tacke1-0/+5
`spack load` and `spack env activate` now use the prefix inspections defined in `modules.yaml`. This allows users to customize/override environment variable modifications if desired. If no `prefix_inspections` configuration is present, Spack uses the values in the default configuration.
2020-11-17spack containerize: allow users to customize the base image (#15028)Massimiliano Culpo13-238/+514
This PR reworks a few attributes in the container subsection of spack.yaml to permit the injection of custom base images when generating containers with Spack. In more detail, users can still specify the base operating system and Spack version they want to use: spack: container: images: os: ubuntu:18.04 spack: develop in which case the generated recipe will use one of the Spack images built on Docker Hub for the build stage and the base OS image in the final stage. Alternatively, they can specify explicitly the two base images: spack: container: images: build: spack/ubuntu-bionic:latest final: ubuntu:18.04 and it will be up to them to ensure their consistency. Additional changes: * This commit adds documentation on the two approaches. * Users can now specify OS packages to install (e.g. with apt or yum) prior to the build (previously this was only available for the finalized image). * Handles to avoid an update of the available system packages have been added to the configuration to facilitate the generation of recipes permitting deterministic builds.
2020-11-17concretizer: modified weights for providers and matching for externalsMassimiliano Culpo6-35/+56
This commit address the case of concretizing a root spec with a transitive conditional dependency on a virtual package, provided by an external. Before these modifications default variant values for the dependency bringing in the virtual package were not respected, and the external package providing the virtual was added to the DAG. The issue stems from two facts: - Selecting a provider has higher precedence than selecting default variants - To ensure that an external is preferred, we used a negative weight To solve it we shift all the providers weight so that: - External providers have a weight of 0 - Non external provider have a weight of 10 or more Using a weight of zero for external providers is such that having an external provider, if present, or not having a provider at all has the same effect on the higher priority minimization. Also fixed a few minor bugs in concretize.lp, that were causing spurious entries in the final answer set. Cleaned concretize.lp from leftover rules.
2020-11-17concretizer: maximize the number of default values used for a single variantMassimiliano Culpo2-0/+17
If a the default of a multi-valued variant is set to multiple values either in package.py or in packages.yaml we need to ensure that all the values are present in the concretized spec. Since each default value has a weight of 0 and the variant value is set implicitly by the concretizer we need to add a rule to maximize on the number of default values that are used.
2020-11-17concretizer: don't require a provider for virtual deps if spec is externalMassimiliano Culpo5-3/+33
This commit introduces a new rule: real_node(Package) :- not external(Package), node(Package). that permits to distinguish between an external node and a real node that shouldn't trim dependency. It solves the case of concretizing ninja with an external Python.
2020-11-17concretizer: spec_clauses() shouldn't emit node_compiler_hard for rule bodies.Todd Gamblin1-2/+0
`node_compiler_hard()` means that something explicitly asked for a node's compiler to be set -- i.e., it's not inherited, it's required. We're generating this in spec_clauses even for specs in rule bodies, which results in conditions like this for optional dependencies: In py-torch/package.py: depends_on('llvm-openmp', when='%apple-clang +openmp') In the generated ASP: declared_dependency("py-torch","llvm-openmp","build") :- node("py-torch"), variant_value("py-torch","openmp","True"), node_compiler("py-torch","apple-clang"), node_compiler_hard("py-torch","apple-clang"), node_compiler_version_satisfies("py-torch","apple-clang",":"). The `node_compiler_hard` there means we would have to *explicitly* set py-torch's compiler to trigger the llvm-openmp dependency, rather than just letting it be set by preferences. This is wrong; the dependency should be there regardless of how the compiler was set. - [x] remove fn.node_compiler_hard() call from spec_clauses when generating rule body clauses.
2020-11-17concretizer: don't generate rules for empty version listsTodd Gamblin1-0/+4
If the version list passed to one_of_iff is empty, it still generates a rule like this: node_compiler_version_satisfies("fujitsu-mpi", "arm", ":") :- 1 { } 1. 1 { } 1 :- node_compiler_version_satisfies("fujitsu-mpi", "arm", ":"). The cardinality rules on the right and left above are never satisfiale, and these rules do nothing. - [x] Skip generating any rules at all for empty version lists.
2020-11-17concretizer: add a rule to avoid cycles in the graph of dependenciesMassimiliano Culpo2-5/+9
2020-11-17External packages have a consistent hash across different concretizersMassimiliano Culpo2-1/+12
2020-11-17Don't fail if MV variants have a tuple as default valueMassimiliano Culpo1-1/+2
2020-11-17Fixup for target preferencesMassimiliano Culpo2-3/+11
2020-11-17Added unit tests to for regressions on open concretizer bugsMassimiliano Culpo11-8/+306
2020-11-17Changed clingo optionsMassimiliano Culpo1-1/+4