summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-05-24ascent package: capture spack global flags in host config (#23826)Cyrus Harrison1-0/+17
2021-05-24dray package: capture spack global flags in host config (#23827)Cyrus Harrison1-0/+17
2021-05-24iwyu: add 0.15 (#23876)Michael Kuhn1-0/+2
2021-05-24Fenicsx packages: updates for 0.1.0 releases (#23836)Chris Richardson6-19/+46
2021-05-24raxml-ng update to version 1.0.2 (#23764)Desmond Orton1-1/+2
* Package update to version 1.0.2 * switched submodule boolean to string * switched from string to bools Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2021-05-24SuperLU: add v5.2.2 and cmake tweaks (#23863)Seth R. Johnson1-26/+39
- Changed to cmake package with backward compatibility with older makefile - Removed unused cmake variable 'blas_blas_libs' - Added new version 5.2.2 which change to external blas variable - Remove unused tcsh dependency - Change URL to use git repository for current and future versions - Add older 4.2 version - Add conflict for older versions with apple-clang
2021-05-24gdk-pixbuf: restrict patch application to meson packages (#23838)Andreas Baumbach1-1/+3
2021-05-24libedit: add missing pkgconfig dependency (#23839)Andreas Baumbach1-0/+1
2021-05-24ncdu: add v1.15.1 (#23840)Andreas Baumbach1-0/+3
2021-05-24alpaka: add new package (#23576)Valentin Volkl1-0/+77
2021-05-24chapel: add versions up to 1.24.1 (#23365)Hang Yan1-1/+7
2021-05-24ucx: add v1.10.1 (#23858)Robert Mijakovic1-1/+2
Co-authored-by: Robert Mijakovic <robert.mijakovic@lxp.lu>
2021-05-24CUDA: add v11.3.1 (#23864)Filippo Spiga1-0/+4
2021-05-24chaplin: add new package (#23866)Valentin Volkl1-0/+22
2021-05-22updating debugedit to be built from its production repository (#23851)Vanessasaurus1-6/+9
Signed-off-by: vsoch <vsoch@users.noreply.github.com> Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2021-05-22Spack can Use RHEL8's `platform-python` if nothing else is available. (#23857)Todd Gamblin2-3/+2
This adds RHEL8's `/usr/libexec/platform-python` to Spack's list of preferred pythons. It will only be used if no other `python` is available in the `PATH`. We have been testing with this python for a while now, and it seems to do all that we need. If Spack one day isn't able to work with it, we'll take it out, but for now it is useful to allow Spack to be used on RHEL8 without a dedicated `python` installation.
2021-05-22Merge tag 'v0.16.2' into developTodd Gamblin1-0/+27
2021-05-22Update CHANGELOG and release version for v0.16.2v0.16.2Todd Gamblin2-1/+28
2021-05-22Style fixes for v0.16.2 releaseMassimiliano Culpo8-12/+22
2021-05-22performance: speed up existence checks in packages (#23661)Todd Gamblin3-24/+37
Spack doesn't require users to manually index their repos; it reindexes the indexes automatically when things change. To determine when to do this, it has to `stat()` all package files in each repository to make sure that indexes up to date with packages. We currently index virtual providers, patches by sha256, and tags on packages. When this was originally implemented, we ran the checker all the time, at startup, but that was slow (see #7587). But we didn't go far enough -- it still consults the checker and does all the stat operations just to see if a package exists (`Repo.exists()`). That might've been a wash in 2018, but as the number of packages has grown, it's gotten slower -- checking 5k packages is expensive and users see this for small operations. It's a win now to make `Repo.exists()` check files directly. **Fix:** This PR does a number of things to speed up `spack load`, `spack info`, and other commands: - [x] Make `Repo.exists()` check files directly again with `os.path.exists()` (this is the big one) - [x] Refactor `Spec.satisfies()` so that a checking for virtual packages only happens if needed (avoids some calls to exists()) - [x] Avoid calling `Repo.exists(spec)` in `Repo.get()`. `Repo.get()` will ultimately try to load a `package.py` file anyway; we can let the failure to load it indicate that the package doesn't exist, and avoid another call to exists(). - [x] Fix up some comments in spec parsing - [x] Call `UnknownPackageError` more consistently in `repo.py`
2021-05-22ASP-based solve: minimize compiler mismatches (#23016)Massimiliano Culpo3-32/+28
fixes #22718 Instead of trying to maximize the number of matches (preferred behavior), try to minimize the number of mismatches (unwanted behavior).
2021-05-22new package for openvdb (#23581)Drew Whitehouse1-0/+90
2021-05-22New package: sparskit (#23848)Olivier Cessenat1-0/+76
2021-05-22ASP-based solver: no intermediate package for concretizing together (#23307)Massimiliano Culpo2-0/+33
The ASP-based solver can natively manage cases where more than one root spec is given, and is able to concretize all the roots together (ensuring one spec per package at most). Modifications: - [x] When concretising together an environment the ASP-based solver calls directly its `solve` method rather than constructing a temporary fake root package.
2021-05-22Import hooks using Python's built-in machinery (#23288)Massimiliano Culpo1-29/+38
The function we coded in Spack to load Python modules with arbitrary names from a file seem to have issues with local imports. For loading hooks though it is unnecessary to use such functions, since we don't care to bind a custom name to a module nor we have to load it from an unknown location. This PR thus modifies spack.hook in the following ways: - Use __import__ instead of spack.util.imp.load_source (this addresses #20005) - Sync module docstring with all the hooks we have - Avoid using memoization in a module function - Marked with a leading underscore all the names that are supposed to stay local
2021-05-22spack location: fix usage without args (#22755)Harmen Stoppels2-4/+5
2021-05-22Add "spack [cd|location] --source-dir" (#22321)Harmen Stoppels3-58/+82
2021-05-22Use Python's built-in machinery to import compilers (#23290)Massimiliano Culpo1-8/+7
2021-05-22ASP-based solver: suppress warnings when constructing facts (#23090)Massimiliano Culpo1-1/+4
fixes #22786 Trying to get optimization flags for a specific target from a compiler may trigger warnings. In the context of constructing facts for the ASP-based solver we don't want to show these warnings to the user, so here we simply ignore them.
2021-05-22Externals with merged prefixes (#22653)Peter Scheibel2-17/+85
We remove system paths from search variables like PATH and from -L options because they may contain many packages and could interfere with Spack-built packages. External packages may be installed to prefixes that are not actually system paths but are still "merged" in the sense that many other packages are installed there. To avoid conflicts, this PR places all external packages at the end of search paths.
2021-05-22ASP-based solver: assign OS correctly with inheritance from parent (#22896)Massimiliano Culpo3-0/+28
fixes #22871 When in presence of multiple choices for the operating system we were lacking a rule to derive the node OS if it was inherited.
2021-05-22"spack build-env" searches env for relevant spec (#21642)Peter Scheibel5-4/+117
If you install packages using spack install in an environment with complex spec constraints, and the install fails, you may want to test out the build using spack build-env; one issue (particularly if you use concretize: together) is that it may be hard to pass the appropriate spec that matches what the environment is attempting to install. This updates the build-env command to default to pulling a matching spec from the environment rather than concretizing what the user provides on the command line independently. This makes a similar change to spack cd. If the user-provided spec matches multiple specs in the environment, then these commands will now report an error and display all matching specs (to help the user specify). Co-authored-by: Gregory Becker <becker33@llnl.gov>
2021-05-22Remove erroneous warnings about quotes for from_source_file (#22767)Adam J. Stewart2-10/+18
2021-05-22Bootstrapping: swap store before configuration (#22631)Massimiliano Culpo3-6/+45
fixes #22294 A combination of the swapping order for global variables and the fact that most of them are lazily evaluated resulted in custom install tree not being taken into account if clingo had to be bootstrapped. This commit fixes that particular issue, but a broader refactor may be needed to ensure that similar situations won't affect us in the future.
2021-05-22Bootstrap: add _builtin config scope (#22610)Harmen Stoppels1-2/+5
(cherry picked from commit a37c916dff5a5c6e72f939433931ab69dfd731bd)
2021-05-22Fix clearing cache of InternalConfigScope (#22609)Harmen Stoppels3-40/+49
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-05-22bugfix for active when pkg is already active error (#22587)Cyrus Harrison1-2/+7
* bugfix for active when pkg is already active error Co-authored-by: Greg Becker <becker33@llnl.gov>
2021-05-22Enforce uniqueness of the version_weight atom per nodeMassimiliano Culpo4-6/+43
fixes #22565 This change enforces the uniqueness of the version_weight atom per node(Package) in the DAG. It does so by applying FTSE and adding an extra layer of indirection with the possible_version_weight/2 atom. Before this change it may have happened that for the same node two different version_weight/2 were in the answer set, each of which referred to a different spec with the same version, and their weights would sum up. This lead to unexpected result like preferring to build a new version of an external if the external version was older.
2021-05-22Externals are preferred even when they have non-default variant valuesMassimiliano Culpo5-2/+64
fixes #22596 Variants which are specified in an external spec are not scored negatively if they encode a non-default value.
2021-05-22clingo: modify recipe for bootstrapping (#22354)Massimiliano Culpo4-16/+83
* clingo: modify recipe for bootstrapping Modifications: - clingo builds with shared Python only if ^python+shared - avoid building the clingo app for bootstrapping - don't link to libpython when bootstrapping * Remove option that breaks on linux * Give more hints for the current Python * Disable CLINGO_BUILD_PY_SHARED for bootstrapping * bootstrapping: try to detect the current python from std library This is much faster than calling external executables * Fix compatibility with Python 2.6 * Give hints on which compiler and OS to use when bootstrapping This change hints which compiler to use for bootstrapping clingo (either GCC or Apple Clang on MacOS). On Cray platforms it also hints to build for the frontend system, where software is meant to be installed. * Use spec_for_current_python to constrain module requirement (cherry picked from commit d5fa509b072f0e58f00eaf81c60f32958a9f1e1d)
2021-05-22ASP-based solver: model disjoint sets for multivalued variants (#22534)Massimiliano Culpo4-2/+64
* ASP-based solver: avoid adding values to variants when they're set fixes #22533 fixes #21911 Added a rule that prevents any value to slip in a variant when the variant is set explicitly. This is relevant for multi-valued variants, in particular for those that have disjoint sets of values. * Ensure disjoint sets have a clear semantics for external packages
2021-05-22Make SingleFileScope able to repopulate the cache after clearing it (#22559)Massimiliano Culpo2-16/+47
fixes #22547 SingleFileScope was not able to repopulate its cache before this change. This was affecting the configuration seen by environments using clingo bootstrapped from sources, since the bootstrapping operation involved a few cache invalidation for config files.
2021-05-22Channelflow: Fix the package. (#22483)RĂ©mi Lacroix1-1/+1
A search and replace went wrong in 2264e30d99d8b9fbdec8fa69b594e53d8ced15a1. Thanks to @wadudmiah who reported this issue.
2021-05-22spack location: bugfix for out of source build dirs (#22348)Harmen Stoppels1-1/+13
2021-05-22bugfix: allow imposed constraints to be overridden in special casesTodd Gamblin1-6/+16
In most cases, we want condition_holds(ID) to imply any imposed constraints associated with the ID. However, the dependency relationship in Spack is special because it's "extra" conditional -- a dependency *condition* may hold, but we have decided that externals will not have dependencies, so we need a way to avoid having imposed constraints appear for nodes that don't exist. This introduces a new rule that says that constraints are imposed *unless* we define `do_not_impose(ID)`. This allows rules like dependencies, which rely on more than just spec conditions, to cancel imposed constraints. We add one special case for this: dependencies of externals.
2021-05-22bugfix: do not generate dep conditions when no dependencyTodd Gamblin1-10/+15
We only consider test dependencies some of the time. Some packages are *only* test dependencies. Spack's algorithm was previously generating dependency conditions that could hold, *even* if there was no potential dependency type. - [x] change asp.py so that this can't happen -- we now only generate dependency types for possible dependencies.
2021-05-22concretizer: unify logic for spec conditionalsTodd Gamblin3-183/+119
This builds on #20638 by unifying all the places in the concretizer where things are conditional on specs. Previously, we duplicated a common spec conditional pattern for dependencies, virtual providers, conflicts, and externals. That was introduced in #20423 and refined in #20507, and roughly looked as follows. Given some directives in a package like: ```python depends_on("foo@1.0+bar", when="@2.0+variant") provides("mpi@2:", when="@1.9:") ``` We handled the `@2.0+variant` and `@1.9:` parts by generating generated `dependency_condition()`, `required_dependency_condition()`, and `imposed_dependency_condition()` facts to trigger rules like this: ```prolog dependency_conditions_hold(ID, Parent, Dependency) :- attr(Name, Arg1) : required_dependency_condition(ID, Name, Arg1); attr(Name, Arg1, Arg2) : required_dependency_condition(ID, Name, Arg1, Arg2); attr(Name, Arg1, Arg2, Arg3) : required_dependency_condition(ID, Name, Arg1, Arg2, Arg3); dependency_condition(ID, Parent, Dependency); node(Parent). ``` And we handled `foo@1.0+bar` and `mpi@2:` parts ("imposed constraints") like this: ```prolog attr(Name, Arg1, Arg2) :- dependency_conditions_hold(ID, Package, Dependency), imposed_dependency_condition(ID, Name, Arg1, Arg2). attr(Name, Arg1, Arg2, Arg3) :- dependency_conditions_hold(ID, Package, Dependency), imposed_dependency_condition(ID, Name, Arg1, Arg2, Arg3). ``` These rules were repeated with different input predicates for requirements (e.g., `required_dependency_condition`) and imposed constraints (e.g., `imposed_dependency_condition`) throughout `concretize.lp`. In #20638 it got to be a bit confusing, because we used the same `dependency_condition_holds` predicate to impose constraints on conditional dependencies and virtual providers. So, even though the pattern was repeated, some of the conditional rules were conjoined in a weird way. Instead of repeating this pattern everywhere, we now have *one* set of consolidated rules for conditions: ```prolog condition_holds(ID) :- condition(ID); attr(Name, A1) : condition_requirement(ID, Name, A1); attr(Name, A1, A2) : condition_requirement(ID, Name, A1, A2); attr(Name, A1, A2, A3) : condition_requirement(ID, Name, A1, A2, A3). attr(Name, A1) :- condition_holds(ID), imposed_constraint(ID, Name, A1). attr(Name, A1, A2) :- condition_holds(ID), imposed_constraint(ID, Name, A1, A2). attr(Name, A1, A2, A3) :- condition_holds(ID), imposed_constraint(ID, Name, A1, A2, A3). ``` this allows us to use `condition(ID)` and `condition_holds(ID)` to encapsulate the conditional logic on specs in all the scenarios where we need it. Instead of defining predicates for the requirements and imposed constraints, we generate the condition inputs with generic facts, and define predicates to associate the condition ID with a particular scenario. So, now, the generated facts for a condition look like this: ```prolog condition(121). condition_requirement(121,"node","cairo"). condition_requirement(121,"variant_value","cairo","fc","True"). imposed_constraint(121,"version_satisfies","fontconfig","2.10.91:"). dependency_condition(121,"cairo","fontconfig"). dependency_type(121,"build"). dependency_type(121,"link"). ``` The requirements and imposed constraints are generic, and we associate them with their meaning via the id. Here, `dependency_condition(121, "cairo", "fontconfig")` tells us that condition 121 has to do with the dependency of `cairo` on `fontconfig`, and the conditional dependency rules just become: ```prolog dependency_holds(Package, Dependency, Type) :- dependency_condition(ID, Package, Dependency), dependency_type(ID, Type), condition_holds(ID). ``` Dependencies, virtuals, conflicts, and externals all now use similar patterns, and the logic for generating condition facts is common to all of them on the python side, as well. The more specific routines like `package_dependencies_rules` just call `self.condition(...)` to get an id and generate requirements and imposed constraints, then they generate their extra facts with the returned id, like this: ```python def package_dependencies_rules(self, pkg, tests): """Translate 'depends_on' directives into ASP logic.""" for _, conditions in sorted(pkg.dependencies.items()): for cond, dep in sorted(conditions.items()): condition_id = self.condition(cond, dep.spec, pkg.name) # create a condition and get its id self.gen.fact(fn.dependency_condition( # associate specifics about the dependency w/the id condition_id, pkg.name, dep.spec.name )) # etc. ``` - [x] unify generation and logic for conditions - [x] use unified logic for dependencies - [x] use unified logic for virtuals - [x] use unified logic for conflicts - [x] use unified logic for externals LocalWords: concretizer mpi attr Arg concretize lp cairo fc fontconfig LocalWords: virtuals def pkg cond dep fn refactor github py
2021-05-22bootstrap: account for platform specific configuration scopes (#22489)Massimiliano Culpo1-7/+18
This change accounts for platform specific configuration scopes, like ~/.spack/linux, during bootstrapping. These scopes were previously not accounted for and that was causing issues e.g. when searching for compilers. (cherry picked from commit 413c422e53843a9e33d7b77a8c44dcfd4bf701be)
2021-05-22Bootstrap clingo from sources (#21446)Massimiliano Culpo12-46/+363
* Allow the bootstrapping of clingo from sources Allow python builds with system python as external for MacOS * Ensure consistent configuration when bootstrapping clingo This commit uses context managers to ensure we can bootstrap clingo using a consistent configuration regardless of the use case being managed. * Github actions: test clingo with bootstrapping from sources * Add command to inspect and clean the bootstrap store Prevent users to set the install tree root to the bootstrap store * clingo: documented how to bootstrap from sources Co-authored-by: Gregory Becker <becker33@llnl.gov> (cherry picked from commit 10e9e142b75c6ca8bc61f688260c002201cc1b22)
2021-05-22Fix makefile filter suggestions (#23856)Seth R. Johnson2-6/+8
Bash has a builtin `fc` that will override the compiler if you use "fc", so it's better to use the full spack-supplied compiler path. Additionally, the filter regex in the docs was wrong: it replaced the entire assignment operation with the RHS.