summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2020-11-17concretizer: add a configuration option to use new or old concretizerTodd Gamblin4-1/+51
- [x] spec.py can call out to the new concretizer - [x] config.yaml now has an option to choose a concretizer (original, clingo)
2020-11-17concretizer: use repository names, not specs with is_virtualTodd Gamblin2-1/+4
2020-11-17concretizer: refactor to support multiple solver backendsTodd Gamblin1-285/+276
There are now three parts: - `SpackSolverSetup` - Spack-specific logic for generating constraints. Calls methods on `AspTextGenerator` to set up the solver with a Spack problem. This shouln't change much from solver backend to solver backend. - ClingoDriver - The solver driver provides methods for SolverSetup to generates an ASP program, send it to `clingo` (run as an external tool), and parse the output into function tuples suitable for `SpecBuilder`. - The interface is generic and should not have to change much for a driver for, say, the Clingo Python interface. - SpecBuilder - Builds Spack specs from function tuples parsed by the solver driver.
2020-11-17concretizer: set spec constraints correctly for body and headTodd Gamblin1-3/+5
2020-11-17concretizer: allow non-default OS, inherit OS along dependenciesTodd Gamblin2-14/+30
2020-11-17tests: add framework to mock targetsTodd Gamblin5-7/+530
2020-11-17concretizer: split platforms, OS, and targets apart in Python and ASPTodd Gamblin2-25/+69
2020-11-17concretizer: targets are inherited like compilersTodd Gamblin1-1/+10
2020-11-17concretizer: change single-letter variables to descriptive namesTodd Gamblin1-108/+175
The original implementation was difficult to read, as it only had single-letter variable names. This converts all of them to descriptive names, e.g., P -> Package, V -> Virtual/Version/Variant, etc.
2020-11-17concretizer: handle compiler existence check settingsTodd Gamblin1-4/+27
To handle unknown compilers propely in tests (and elsewhere), we need to add unknown compilers from the spec to the list of possible compilers. Rework how the compiler list is generated and includes compilers from specs if the existence check is disabled.
2020-11-17concretizer: add initial package existence checkTodd Gamblin1-0/+12
2020-11-17concretizer: handle virtual spec constraints betterTodd Gamblin2-4/+11
Specs like hdf5 ^mpi were unsatisfiable because we added a requierment for `node("mpi").`. This can't be resolved because "mpi" is not a package. - [x] Introduce `virtual_node()`, which says *some* provider must be in the DAG.
2020-11-17concretizer: solve with compiler flags but preserve orderTodd Gamblin3-14/+166
This adds compiler flags to the ASP solve so that we can have conditions based on them in the solve. But, it keeps order out of the solve to avoid unneeded complexity and combinatorial explosions. The solver determines which flags are on a spec, but the order is determined by DAG precedence (childrens' flags take precedence over parents' and are added on the right) and order (order flags were specified on the command line is respected). The solver is responsible for determining when to propagate flags, when to inheit them from other nodes, when to take them from compiler preferences, etc.
2020-11-17concretizer: add timers around phasesTodd Gamblin2-4/+38
2020-11-17concretizer: optimize microarchitectures, constrained by compiler supportTodd Gamblin3-66/+145
Weight microarchitectures and prefers more rercent ones. Also disallow nodes where the compiler does not support the selected target. We should revisit this at some point as it seems like if I play around with the compiler support for different architectures, the solver runs very slowly. See notes in comments -- the bad case was gcc supporting broadwell and skylake with clang maxing out at haswell.
2020-11-17concretizer bugfix: require at least one value for multi-value variantsTodd Gamblin1-0/+4
We didn't have a cardinality constraint for multi-valued variants, so the solver wasn't filling them in. - [x] add a requirement for at least one value for multi-valued variants
2020-11-17commands: add --json argument to `spack solve`Todd Gamblin1-10/+18
2020-11-17concretizer: make some rules into factsTodd Gamblin1-9/+5
2020-11-17concretizer bugfix: all variants need possible valuesTodd Gamblin1-0/+4
Variants like `cpu_target` on `openblas` don't have defineed values, but they have a default. Ensure that the default is always a possible value for the solver.
2020-11-17concretizer bugfix: fix generations of conditionals for dependenciesTodd Gamblin1-4/+7
Spack was generating the same dependency connstraints twice in the output ASP: ``` declared_dependency("abinit", "hdf5", "link") :- node("abinit"), variant_value("abinit", "mpi", "True"), variant_value("abinit", "mpi", "True"). ``` This was because `AspFunction` was modifying itself when called. - [x] fix `AspFunction` so that every call returns a new object
2020-11-17concretizer bugfix: *at most* one provider for any virtualTodd Gamblin1-2/+2
2020-11-17concretizer: optimized for preferred virtuals before recent versionsTodd Gamblin2-5/+5
2020-11-17concretizer: handle compiler preferences with optimizationTodd Gamblin4-73/+126
- [x] Add support for packages.yaml and command-line compiler preferences. - [x] Rework compiler version propagation to use optimization rather than hard logic constraints
2020-11-17concretizer: deterministic order for asp output for better diffsTodd Gamblin1-14/+14
Technically the ASP output order does not matter, but it's hard to diff two different solve fomulations unless we order it. - [x] make sure ASP output is emitted in a deterministic order (by sorting all hash keys)
2020-11-17concretizer: rename --dump to --showTodd Gamblin1-8/+8
2020-11-17concretizer: handle package namespacesTodd Gamblin1-0/+7
2020-11-17concretizer: handle constraints on dependencies, adjust optimizationTodd Gamblin3-10/+49
This needs more thought, as I am pretty sure the weights are not correct. Or, at least, I'm not convinced that they do what we want in all cases. See note in concretize.lp.
2020-11-17concretizer: handle dependency typesTodd Gamblin5-17/+42
2020-11-17concretizer: prioritize versions by package pref, newest, preferred, actualTodd Gamblin2-2/+53
Solver now prefers newer versions like the old concretizer. Prefer package preferences from packages.yaml, preferred=True, package definition, and finally each version itself.
2020-11-17concretizer: Use "competition" output format to avoid extra parsingTodd Gamblin2-18/+45
Competition output only prints out one model, so we do not have to unnecessarily parse all the non-optimal models. We'll just look at the best model and bring that in. In practice, this saves a lot of JSON parsing and spec construction time.
2020-11-17concretizer: handle virtual provider preferences from packages.yamlTodd Gamblin2-11/+83
2020-11-17concretizer: use clingo json output instead of textTodd Gamblin2-69/+85
Clingo actually has an option to output JSON -- use that instead of parsing the raw otuput ourselves. This also allows us to pick the best answer -- modify the parser to *only* construct a spec for that one rather than building all of them like we did before.
2020-11-17concretizer: require only one provider for any virtual in the DAGTodd Gamblin2-4/+8
2020-11-17concretizer: handle variant defaults with optimizationTodd Gamblin2-9/+49
- Instead of using default logic, handle variant defaults by minimizing the number of non-default variants in the solution. - This actually seems to be pretty fast, and it fixes the long-standing issue that writing this: spack install hdf5 ^mpich will fail if you don't specify hdf5+mpi. With optimization and allowing enums to be enumerated, the solver seems to be able to quickly discover that +mpi is the only way hdf5 can depend on mpich, and it forces the switch to be thrown.
2020-11-17concretizer: support conditional dependenciesTodd Gamblin2-16/+63
2020-11-17variants: allow MultiValuedVariants to be constructed incrementallyTodd Gamblin2-5/+18
2020-11-17concretizer: initial support for virtual dependenciesTodd Gamblin2-6/+18
Add initial support for virtual dependencies. Solver now knows about all virtuals and can choose one to resolve a dependency.
2020-11-17concretizer: print out virtualsTodd Gamblin2-2/+12
2020-11-17concretizer: handle versions with choice construct rather than conflictsTodd Gamblin1-40/+95
Use '1 { version(x); version(y); version(z) } 1.' instead of declaring conflicts for non-matching versions. This keeps the sense of version clauses positive, which will allow them to be used more easily in conditionals later. Also refactor `spec_clauses()` method to return clauses that can be used in conditions, etc. instead of just printing out facts.
2020-11-17concretizer: add another definition pragma.Todd Gamblin1-0/+1
- single_value_variant may not be defined by the generated program. Mark it to avoid warnings.
2020-11-17concretizer: cleanupTodd Gamblin1-14/+17
2020-11-17concretizer: use conditional literals for versions.Todd Gamblin2-24/+31
2020-11-17concretizer: mark depends_on/2 defined for solves without dependencies.Todd Gamblin1-0/+3
2020-11-17concretizer: add basic semantics for compilersTodd Gamblin3-7/+112
- This handles setting the compiler and falling back to a default compiler, as well as providing default values for compilers/compiler versions. - Versions still aren't quite right -- you can't properly override versions on compiler specs.
2020-11-17concretizer: simplify and move architecture semantics into concretize.lpTodd Gamblin2-15/+41
- Model architecture default settings and propagation off of variants - Leverage ASP default logic to set architecture to default if it's not set otherwise. - Move logic out of Python and into concretize.lp as first-order rules.
2020-11-17concretizer: break output up into easier-to-understand sectionsTodd Gamblin3-22/+37
2020-11-17concretizer: simplify and suppress warnings for variant handlingTodd Gamblin2-8/+19
We are relying on default logic in the variant handling in that we set a default value if we never see `variant_set(P, V, X)`. - Move the logic for this into `concretize.lp` instead of generating it for every package. - For programs that don't have explicit variant settings, clingo warns that variant_set(P, V, X) doesn't appear in any rule head, because a setting is never generated. - Specifically suppress this warning.
2020-11-17concretizer: split long lines in ASP programsTodd Gamblin1-1/+8
2020-11-17concretizer: split main logic program out into filesTodd Gamblin3-56/+57
- Add `concretize.lp` and `display.lp` as independent files - Dump them instead of embedded strings
2020-11-17concretizer: colorize ASP outputTodd Gamblin1-7/+46