summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2020-11-17concretizer: handle dependencies conditional on other dependenciesMassimiliano Culpo1-5/+10
2020-11-17tests: verify to handle dependencies conditional on other dependenciesMassimiliano Culpo1-0/+18
2020-11-17concretizer: handle conflicts with compiler ranges correctlyMassimiliano Culpo3-20/+54
As reported, conflicts with compiler ranges were not treated correctly. This commit adds tests to verify the expected behavior for the new concretizer. The new rules to enforce a correct behavior involve: - Adding a rule to prefer the compiler selected for the root package, if no other preference is set - Give a strong negative weight to compiler preferences expressed in packages.yaml - Maximize on compiler AND compiler version match
2020-11-17Github actions: add CI for ASP based solverMassimiliano Culpo3-0/+16
2020-11-17Make all tests passMassimiliano Culpo8-28/+41
Fixed a couple of tests and marked a few xfails to solve them later.
2020-11-17concretizer: added handling for dev_path variantMassimiliano Culpo3-10/+77
This variant is currently either set from command line, in which case it enters the concretization, or attached from environment after concretization.
2020-11-17concretizer: ensure upfront that variants are validMassimiliano Culpo3-12/+31
2020-11-17concretizer: account for test dependencies only when requiredMassimiliano Culpo3-39/+38
2020-11-17Fix installer.py unit tests that check outputMassimiliano Culpo1-1/+2
2020-11-17Compute the correct package name for hierarchies that change class namesMassimiliano Culpo1-1/+5
2020-11-17concretizer: handle variants defined through validatorsMassimiliano Culpo2-2/+23
Variant of this kind don't have a list of possible values encoded in the ASP facts. Since all we have is a validator the list of possible values just includes just the default value and possibly the value passed from packages.yaml or cli.
2020-11-17concretizer: account for patches variantMassimiliano Culpo2-44/+55
This is done after the builder has actually built the specs, to respect the semantics use with the old concretizer. Later we could move this to the solver as a multivalued variant.
2020-11-17concretizer: ensure that no deprecated spec is being usedMassimiliano Culpo2-16/+30
This is done after the builder has actually built the specs, to respect the semantics use with the old concretizer. A better approach is to substitute the spec directly in concretization.
2020-11-17conftest: hook the new solver in the config fixtureMassimiliano Culpo6-10/+12
2020-11-17concretizer: handle "none" value and '*' wildcardMassimiliano Culpo2-2/+9
The "none" variant value cannot be combined with other values. The '*' wildcard matches anything, including "none". It's thus relevant in queries, but disregarded in concretization.
2020-11-17Fixed failing unit testsMassimiliano Culpo6-20/+27
- The test on concretization of anonymous dependencies has been fixed by raising the expected exception. - The test on compiler bootstrap has been fixed by updating the version of GCC used in the test. Since gcc@2.0 does not support targets later than x86_64, the new concretizer was looking for a non-existing spec, i.e. it was correctly trying to retrieve 'gcc target=x86_64' instead of 'gcc target=core2'. - The test on gitlab CI needed an update of the target
2020-11-17concretizer: virtual entry in packages.yaml, external modulesMassimiliano Culpo2-11/+54
This commit adds support for specifying rules in packages.yaml that refer to virtual packages. The approach is to normalize in memory each configuration and turn it into an equivalent configuration without rules on virtual. This is possible if the set of packages to be handled is considered fixed.
2020-11-17concretizer: concretize a virtual rootMassimiliano Culpo4-5/+41
Before this modification the root of a DAG has to be a real package. This commit adds rules to concretize virtual roots.
2020-11-17concretizer: handle version preferences from packages.yamlMassimiliano Culpo2-2/+22
2020-11-17concretizer: handle target preferences from packages.yamlMassimiliano Culpo2-9/+45
The weight of the target used in concretization is, in order: 1. A specific per package weight, if set in packages.yaml 2. Inherited from the parent, if possible 3. The default target weight (always set)
2020-11-17concretizer: fixed test on compiler preferencesMassimiliano Culpo3-3/+3
2020-11-17concretizer: added logic for preferred variantsMassimiliano Culpo3-20/+58
If preferred variants are present, they'll set the default value of a variant. Otherwise the default value is what is encoded in package.py
2020-11-17concretizer: refine compiler logicMassimiliano Culpo4-17/+42
Concrete versions for compilers are respected verbatim. Permit to use a non-existing compiler if the appropriate configuration option has been set.
2020-11-17Fixed failing unit testsMassimiliano Culpo2-10/+11
- Tests based on TestArchitecture - Tests on non-buildable external
2020-11-17concretizer: prefer using the same compiler over using newer versionsMassimiliano Culpo1-5/+5
2020-11-17concretizer: added support for versioned virtual specsMassimiliano Culpo2-29/+96
2020-11-17concretizer: added rules and code for externalsMassimiliano Culpo4-28/+178
Generate facts on externals by inspecting packages.yaml. Added rules in concretize.lp Added extra logic so that external specs disregard any conflict encoded in the package. In ASP this would be a simple addition to an integrity constraint: :- c1, c2, c3, not external(pkg) Using the the Backend API from Python it requires some scaffolding to obtain a default negated statement.
2020-11-17package_sanity: fixed wrong string formatMassimiliano Culpo1-1/+1
2020-11-17concretizer: add conflict rules from packagesMassimiliano Culpo3-28/+72
Conflict rules from packages are added as integrity constraints in the ASP formulation. Most of the code to generate them has been reused from PyclingoDriver.rules
2020-11-17test_noversion_pkg: generalized the error to be caughtMassimiliano Culpo1-2/+2
The new concretizer and the old concretizer solve constraints in a different way. Here we ensure that a SpackError is raised, instead of a specific error that made sense in the old concretizer but probably not in the new.
2020-11-17compiler constraints: deduplicate the list of compilers before encoding ↵Massimiliano Culpo1-0/+1
one_of_iff rules This fixes 8 bugs in test/concretize.py
2020-11-17concretizer: add compiler version constraintsTodd Gamblin6-61/+59
Add rules to account for compiler version constraints in concretize.lp.
2020-11-17concretizer: use cardinality constraints for versionsTodd Gamblin4-42/+60
Instead of python callbacks, use cardinality constraints for package versions. This is slightly faster and has the advantage that it can be written to an ASP program to be executed *outside* of Spack. We can use this in the future to unify the pyclingo driver and the clingo text driver. This makes use of add_weight_rule() to implement cardinality constraints. add_weight_rule() only has a lower bound parameter, but you can implement a strict "exactly one of" constraint using it. In particular, wee want to define: 1 {v1; v2; v3; ...} 1 :- version_satisfies(pkg, constraint). version_satisfies(pkg, constraint) :- 1 {v1; v2; v3; ...} 1. And we do that like this, for every version constraint: atleast1(pkg, constr) :- 1 {version(pkg, v1); version(pkg, v2); ...}. morethan1(pkg, constr) :- 2 {version(pkg, v1); version(pkg, v2); ...}. version_satisfies(pkg, constr) :- atleast1, not morethan1(pkg, constr). :- version_satisfies(pkg, constr), morethan1. :- version_satisfies(pkg, constr), not atleast1. v1, v2, v3, etc. are computed on the Python side by comparing every possible package version with the constraint. Computing things like this has the added advantage that if v1, v2, v3, etc. comprise *all* possible versions of a package, we can just omit the rules for the constraint under consideration. This happens pretty frequently in the Spack mainline.
2020-11-17concretizer: first working version with pyclingo interfaceTodd Gamblin4-63/+300
- [x] Solver now uses the Python interface to clingo - [x] can extract unsatisfiable cores from problems when things go wrong - [x] use Python callbacks for versions instead of choice rules (this may ultimately hurt performance)
2020-11-17concretizer: add a configuration option to use new or old concretizerTodd Gamblin3-1/+35
- [x] spec.py can call out to the new concretizer - [x] config.yaml now has an option to choose a concretizer (original, clingo)
2020-11-17concretizer: use repository names, not specs with is_virtualTodd Gamblin2-1/+4
2020-11-17concretizer: refactor to support multiple solver backendsTodd Gamblin1-285/+276
There are now three parts: - `SpackSolverSetup` - Spack-specific logic for generating constraints. Calls methods on `AspTextGenerator` to set up the solver with a Spack problem. This shouln't change much from solver backend to solver backend. - ClingoDriver - The solver driver provides methods for SolverSetup to generates an ASP program, send it to `clingo` (run as an external tool), and parse the output into function tuples suitable for `SpecBuilder`. - The interface is generic and should not have to change much for a driver for, say, the Clingo Python interface. - SpecBuilder - Builds Spack specs from function tuples parsed by the solver driver.
2020-11-17concretizer: set spec constraints correctly for body and headTodd Gamblin1-3/+5
2020-11-17concretizer: allow non-default OS, inherit OS along dependenciesTodd Gamblin2-14/+30
2020-11-17tests: add framework to mock targetsTodd Gamblin5-7/+530
2020-11-17concretizer: split platforms, OS, and targets apart in Python and ASPTodd Gamblin2-25/+69
2020-11-17concretizer: targets are inherited like compilersTodd Gamblin1-1/+10
2020-11-17concretizer: change single-letter variables to descriptive namesTodd Gamblin1-108/+175
The original implementation was difficult to read, as it only had single-letter variable names. This converts all of them to descriptive names, e.g., P -> Package, V -> Virtual/Version/Variant, etc.
2020-11-17concretizer: handle compiler existence check settingsTodd Gamblin1-4/+27
To handle unknown compilers propely in tests (and elsewhere), we need to add unknown compilers from the spec to the list of possible compilers. Rework how the compiler list is generated and includes compilers from specs if the existence check is disabled.
2020-11-17concretizer: add initial package existence checkTodd Gamblin1-0/+12
2020-11-17concretizer: handle virtual spec constraints betterTodd Gamblin2-4/+11
Specs like hdf5 ^mpi were unsatisfiable because we added a requierment for `node("mpi").`. This can't be resolved because "mpi" is not a package. - [x] Introduce `virtual_node()`, which says *some* provider must be in the DAG.
2020-11-17concretizer: solve with compiler flags but preserve orderTodd Gamblin3-14/+166
This adds compiler flags to the ASP solve so that we can have conditions based on them in the solve. But, it keeps order out of the solve to avoid unneeded complexity and combinatorial explosions. The solver determines which flags are on a spec, but the order is determined by DAG precedence (childrens' flags take precedence over parents' and are added on the right) and order (order flags were specified on the command line is respected). The solver is responsible for determining when to propagate flags, when to inheit them from other nodes, when to take them from compiler preferences, etc.
2020-11-17concretizer: add timers around phasesTodd Gamblin2-4/+38
2020-11-17concretizer: optimize microarchitectures, constrained by compiler supportTodd Gamblin3-66/+145
Weight microarchitectures and prefers more rercent ones. Also disallow nodes where the compiler does not support the selected target. We should revisit this at some point as it seems like if I play around with the compiler support for different architectures, the solver runs very slowly. See notes in comments -- the bad case was gcc supporting broadwell and skylake with clang maxing out at haswell.
2020-11-17concretizer bugfix: require at least one value for multi-value variantsTodd Gamblin1-0/+4
We didn't have a cardinality constraint for multi-valued variants, so the solver wasn't filling them in. - [x] add a requirement for at least one value for multi-valued variants
generated by cgit v1.2.3-60-g2f50 (git 2.42.0) at 2024-08-03 04:38:36 +0000