summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2020-11-27concretizer: treat target ranges in directives correctly (#19988)Massimiliano Culpo2-1/+59
fixes #19981 This commit adds support for target ranges in directives, for instance: conflicts('+foo', when='target=x86_64:,aarch64:') If any target in a spec body is not a known target the following clause will be emitted: node_target_satisfies(Package, TargetConstraint) when traversing the spec and a definition of the clause will then be printed at the end similarly to what is done for package and compiler versions.
2020-11-26archspec: added support for aocc (#20124)Massimiliano Culpo3-2/+145
2020-11-26concretizer: prioritize matching compilers over newer versions (#20020)Massimiliano Culpo3-6/+32
fixes #20019 Before this modification having a newer version of a node came at higher priority in the optimization than having matching compilers. This could result in unexpected configurations for packages with conflict directives on compilers of the type: conflicts('%gcc@X.Y:', when='@:A.B') where changing the compiler for just that node is preferred to lower the node version to less than 'A.B'. Now the priority has been switched so the solver will try to lower the version of the nodes in question before changing their compiler.
2020-11-26concretizer: allow a bool to be passed as argument for tests dependencies ↵Massimiliano Culpo3-8/+54
(#20082) refers #20079 Added docstrings to 'concretize' and 'concretized' to document the format for tests. Added tests for the activation of test dependencies.
2020-11-25concretizer: treat conditional providers correctly (#20086)Massimiliano Culpo2-1/+18
refers #20040 This modification emits rules like: provides_virtual("netlib-lapack","blas") :- variant_value("netlib-lapack","external-blas","False"). for packages that provide virtual dependencies conditionally instead of a fact that doesn't account for the condition.
2020-11-25concretizer: remove debug statement (#20085)Harmen Stoppels1-1/+0
2020-11-23Docs: remove duplication in Command Reference (#20021)Adam J. Stewart2-2/+2
2020-11-23recognize macOS 11.1 as big sur (#20038)Martin Aumüller1-2/+6
Big Sur versions go 11.0, 11.0.1, 11.1 (vs. prior versions that only used the minor component) Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2020-11-19spack debug report: print concretizer (#19983)Adam J. Stewart2-0/+4
2020-11-19fujitsu compiler: added / fixed support for compiler flags (#19967)Tomoki, Karatsu2-2/+12
Added flags for: - Debug symbols - C++17 standard Fixed the list of flags for generic optimizations
2020-11-19clang/llvm: fix version detection (#19978)Michael Kuhn2-4/+8
This PR fixes two problems with clang/llvm's version detection. clang's version output looks like this: ``` clang version 11.0.0 Target: x86_64-unknown-linux-gnu ``` This caused clang's version to be misdetected as: ``` clang@11.0.0 Target: ``` This resulted in errors when trying to actually use it as a compiler. When using `spack external find`, we couldn't determine the compiler version, resulting in errors like this: ``` ==> Warning: "llvm@11.0.0+clang+lld+lldb" has been detected on the system but will not be added to packages.yaml [reason=c compiler not found for llvm@11.0.0+clang+lld+lldb] ``` Changing the regex to only match until the end of the line fixes these problems. Fixes: #19473
2020-11-18fix error handling for spack test results command (#19987)Greg Becker1-0/+1
2020-11-18hip support for umpire, chai, raja, camp (#19715)Danny Taller2-0/+139
* create HipPackage base class and do some refactoring * comments and added conflict to raja for openmp with hip
2020-11-18bump version number to 0.16.0Todd Gamblin1-1/+1
2020-11-18cmd: add `spack mark` command (#16662)Michael Kuhn7-33/+329
This adds a new `mark` command that can be used to mark packages as either explicitly or implicitly installed. Apart from fixing the package database after installing a dependency manually, it can be used to implement upgrade workflows as outlined in #13385. The following commands demonstrate how the `mark` and `gc` commands can be used to only keep the current version of a package installed: ```console $ spack install pkgA $ spack install pkgB $ git pull # Imagine new versions for pkgA and/or pkgB are introduced $ spack mark -i -a $ spack install pkgA $ spack install pkgB $ spack gc ``` If there is no new version for a package, `install` will simply mark it as explicitly installed and `gc` will not remove it. Co-authored-by: Greg Becker <becker33@llnl.gov>
2020-11-18spack test (#15702)Greg Becker48-620/+2283
Users can add test() methods to their packages to run smoke tests on installations with the new `spack test` command (the old `spack test` is now `spack unit-test`). spack test is environment-aware, so you can `spack install` an environment and then run `spack test run` to run smoke tests on all of its packages. Historical test logs can be perused with `spack test results`. Generic smoke tests for MPI implementations, C, C++, and Fortran compilers as well as specific smoke tests for 18 packages. Inside the test method, individual tests can be run separately (and continue to run best-effort after a test failure) using the `run_test` method. The `run_test` method encapsulates finding test executables, running and checking return codes, checking output, and error handling. This handles the following trickier aspects of testing with direct support in Spack's package API: - [x] Caching source or intermediate build files at build time for use at test time. - [x] Test dependencies, - [x] packages that require a compiler for testing (such as library only packages). See the packaging guide for more details on using Spack testing support. Included is support for package.py files for virtual packages. This does not change the Spack interface, but is a major change in internals. Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov> Co-authored-by: wspear <wjspear@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-11-17Improve warning message for deprecated attributes in "packages.yaml"Massimiliano Culpo6-28/+67
The deprecatedProperties custom validator now can accept a function to compute a better error message. Improve error/warning message for deprecated properties
2020-11-17Documentation: spack load/environments prefix inspections (#19961)Peter Scheibel3-4/+34
As of #18260, `spack load` and `spack env activate` now use `prefix_inspections` from the modules configuration to decide how to modify environment variables. This updates the modules configuration documentation to describe how to update environment variables with the `prefix_inspections` section. This also updates the `spack load` and environments documentation to refer to the new `prefix_inspections` documentation.
2020-11-17spack load/environments: allow customization of prefix inspections (#18260)Dr. Christian Tacke1-0/+5
`spack load` and `spack env activate` now use the prefix inspections defined in `modules.yaml`. This allows users to customize/override environment variable modifications if desired. If no `prefix_inspections` configuration is present, Spack uses the values in the default configuration.
2020-11-17spack containerize: allow users to customize the base image (#15028)Massimiliano Culpo11-219/+479
This PR reworks a few attributes in the container subsection of spack.yaml to permit the injection of custom base images when generating containers with Spack. In more detail, users can still specify the base operating system and Spack version they want to use: spack: container: images: os: ubuntu:18.04 spack: develop in which case the generated recipe will use one of the Spack images built on Docker Hub for the build stage and the base OS image in the final stage. Alternatively, they can specify explicitly the two base images: spack: container: images: build: spack/ubuntu-bionic:latest final: ubuntu:18.04 and it will be up to them to ensure their consistency. Additional changes: * This commit adds documentation on the two approaches. * Users can now specify OS packages to install (e.g. with apt or yum) prior to the build (previously this was only available for the finalized image). * Handles to avoid an update of the available system packages have been added to the configuration to facilitate the generation of recipes permitting deterministic builds.
2020-11-17concretizer: modified weights for providers and matching for externalsMassimiliano Culpo4-35/+29
This commit address the case of concretizing a root spec with a transitive conditional dependency on a virtual package, provided by an external. Before these modifications default variant values for the dependency bringing in the virtual package were not respected, and the external package providing the virtual was added to the DAG. The issue stems from two facts: - Selecting a provider has higher precedence than selecting default variants - To ensure that an external is preferred, we used a negative weight To solve it we shift all the providers weight so that: - External providers have a weight of 0 - Non external provider have a weight of 10 or more Using a weight of zero for external providers is such that having an external provider, if present, or not having a provider at all has the same effect on the higher priority minimization. Also fixed a few minor bugs in concretize.lp, that were causing spurious entries in the final answer set. Cleaned concretize.lp from leftover rules.
2020-11-17concretizer: maximize the number of default values used for a single variantMassimiliano Culpo2-0/+17
If a the default of a multi-valued variant is set to multiple values either in package.py or in packages.yaml we need to ensure that all the values are present in the concretized spec. Since each default value has a weight of 0 and the variant value is set implicitly by the concretizer we need to add a rule to maximize on the number of default values that are used.
2020-11-17concretizer: don't require a provider for virtual deps if spec is externalMassimiliano Culpo4-3/+17
This commit introduces a new rule: real_node(Package) :- not external(Package), node(Package). that permits to distinguish between an external node and a real node that shouldn't trim dependency. It solves the case of concretizing ninja with an external Python.
2020-11-17concretizer: spec_clauses() shouldn't emit node_compiler_hard for rule bodies.Todd Gamblin1-2/+0
`node_compiler_hard()` means that something explicitly asked for a node's compiler to be set -- i.e., it's not inherited, it's required. We're generating this in spec_clauses even for specs in rule bodies, which results in conditions like this for optional dependencies: In py-torch/package.py: depends_on('llvm-openmp', when='%apple-clang +openmp') In the generated ASP: declared_dependency("py-torch","llvm-openmp","build") :- node("py-torch"), variant_value("py-torch","openmp","True"), node_compiler("py-torch","apple-clang"), node_compiler_hard("py-torch","apple-clang"), node_compiler_version_satisfies("py-torch","apple-clang",":"). The `node_compiler_hard` there means we would have to *explicitly* set py-torch's compiler to trigger the llvm-openmp dependency, rather than just letting it be set by preferences. This is wrong; the dependency should be there regardless of how the compiler was set. - [x] remove fn.node_compiler_hard() call from spec_clauses when generating rule body clauses.
2020-11-17concretizer: don't generate rules for empty version listsTodd Gamblin1-0/+4
If the version list passed to one_of_iff is empty, it still generates a rule like this: node_compiler_version_satisfies("fujitsu-mpi", "arm", ":") :- 1 { } 1. 1 { } 1 :- node_compiler_version_satisfies("fujitsu-mpi", "arm", ":"). The cardinality rules on the right and left above are never satisfiale, and these rules do nothing. - [x] Skip generating any rules at all for empty version lists.
2020-11-17concretizer: add a rule to avoid cycles in the graph of dependenciesMassimiliano Culpo2-5/+9
2020-11-17External packages have a consistent hash across different concretizersMassimiliano Culpo2-1/+12
2020-11-17Don't fail if MV variants have a tuple as default valueMassimiliano Culpo1-1/+2
2020-11-17Fixup for target preferencesMassimiliano Culpo2-3/+11
2020-11-17Added unit tests to for regressions on open concretizer bugsMassimiliano Culpo2-6/+162
2020-11-17Changed clingo optionsMassimiliano Culpo1-1/+4
2020-11-17Reworked optimization rulesMassimiliano Culpo1-15/+49
2020-11-17concretizer: set target preference for inheritance from rootMassimiliano Culpo1-0/+7
2020-11-17install: one less concretization when installing from fileMassimiliano Culpo1-2/+3
2020-11-17Fixed branch after rebase (port to archspec)Massimiliano Culpo2-13/+14
TODO: Investigate the need to remove memoization on Spec.patches (infinite recursion when testing `__contains__`)
2020-11-17Add unit tests for dependencies being patched by parentMassimiliano Culpo1-0/+17
2020-11-17concretizer: handle dependencies conditional on other dependenciesMassimiliano Culpo1-5/+10
2020-11-17tests: verify to handle dependencies conditional on other dependenciesMassimiliano Culpo1-0/+18
2020-11-17concretizer: handle conflicts with compiler ranges correctlyMassimiliano Culpo3-20/+54
As reported, conflicts with compiler ranges were not treated correctly. This commit adds tests to verify the expected behavior for the new concretizer. The new rules to enforce a correct behavior involve: - Adding a rule to prefer the compiler selected for the root package, if no other preference is set - Give a strong negative weight to compiler preferences expressed in packages.yaml - Maximize on compiler AND compiler version match
2020-11-17Github actions: add CI for ASP based solverMassimiliano Culpo3-0/+16
2020-11-17Make all tests passMassimiliano Culpo8-28/+41
Fixed a couple of tests and marked a few xfails to solve them later.
2020-11-17concretizer: added handling for dev_path variantMassimiliano Culpo3-10/+77
This variant is currently either set from command line, in which case it enters the concretization, or attached from environment after concretization.
2020-11-17concretizer: ensure upfront that variants are validMassimiliano Culpo3-12/+31
2020-11-17concretizer: account for test dependencies only when requiredMassimiliano Culpo3-39/+38
2020-11-17Fix installer.py unit tests that check outputMassimiliano Culpo1-1/+2
2020-11-17Compute the correct package name for hierarchies that change class namesMassimiliano Culpo1-1/+5
2020-11-17concretizer: handle variants defined through validatorsMassimiliano Culpo2-2/+23
Variant of this kind don't have a list of possible values encoded in the ASP facts. Since all we have is a validator the list of possible values just includes just the default value and possibly the value passed from packages.yaml or cli.
2020-11-17concretizer: account for patches variantMassimiliano Culpo2-44/+55
This is done after the builder has actually built the specs, to respect the semantics use with the old concretizer. Later we could move this to the solver as a multivalued variant.
2020-11-17concretizer: ensure that no deprecated spec is being usedMassimiliano Culpo2-16/+30
This is done after the builder has actually built the specs, to respect the semantics use with the old concretizer. A better approach is to substitute the spec directly in concretization.
2020-11-17conftest: hook the new solver in the config fixtureMassimiliano Culpo6-10/+12