summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2021-11-23Remove support for Python 2.6 (#27256)Massimiliano Culpo25-288/+62
Modifications: - [x] Removed `centos:6` unit test, adjusted vermin checks - [x] Removed backport of `collections.OrderedDict` - [x] Removed backport of `functools.total_ordering` - [x] Removed Python 2.6 specific skip markers in unit tests - [x] Fixed a few minor Python 2.6 related TODOs in code Updating the vendored dependencies will be done in separate PRs
2021-11-22bugfix: Allow legacy tests to be read after hash break (#26078)Nathan Hanford2-1/+33
* added a test case Co-authored-by: Nathan Hanford <hanford1@llnl.gov>
2021-11-22Intel packages: add support for LLVM OpenMP (#26517)Piotr Luszczek1-1/+7
2021-11-22Make CUDA and ROCm architecture conditional (#27185)Massimiliano Culpo2-2/+4
* Make CUDA and ROCm architecture conditional fixes #14337 The variant to specify which architecture to use for CUDA and ROCm are now conditional on +cuda and +rocm respectively. * cp2k: make all CUDA related variants conditional on +cuda
2021-11-22Make _enable_or_disable(...) return an empty array for conditional variants ↵Harmen Stoppels3-0/+32
whose condition is not met (#27504)
2021-11-19Add connection specification to mirror creation (#24244)Joseph Snyder11-43/+218
* Add connection specification to mirror creation This allows each mirror to contain information about the credentials used to access it. Update command and tests based on comments Switch to only "long form" flags for the s3 connection information. Use the "any" function instead of checking for an empty list when looking for s3 connection information. Split test to use the access token separately from the access id and key. Use long flag form in test. Add endpoint_url to available S3 options. Extend the special parameters for an S3 mirror to accept the endpoint_url parameter. Add a test. * Add connection information per URL not per mirror Expand the mirror-based connection information to be per-URL. This will allow a user to specify different S3 connection information for both the fetch and the push URLs. Add a parameter for "profile", another way of storing the id/secret pair. * Switch from "access_profile" to "profile"
2021-11-19define_from_variant: return an empty string for non-existing variants (#27503)Harmen Stoppels3-0/+29
This permits to use conditional variants without a lot of boilerplate.
2021-11-19Adding --reuse to dev-build command. (#27487)Michael Davis1-2/+2
2021-11-19Remove spurious debug print (#27541)Massimiliano Culpo1-1/+0
2021-11-18Refactor bootstrap of "spack style" dependencies (#27485)Massimiliano Culpo4-156/+156
Remove the "get_executable" function from the spack.bootstrap module. Now "flake8", "isort", "mypy" and "black" will use the same bootstrapping method as GnuPG.
2021-11-18Allow recent pytest versions to be used with Spack (#25371)Massimiliano Culpo94-28/+76
Currently Spack vendors `pytest` at a version which is three major versions behind the latest (3.2.5 vs. 6.2.4). We do that since v3.2.5 is the latest version supporting Python 2.6. Remaining so much behind the currently supported versions though might introduce some incompatibilities and is surely a technical debt. This PR modifies Spack to: - Use the vendored `pytest@3.2.5` only as a fallback solution, if the Python interpreter used for Spack doesn't provide a newer one - Be able to parse `pytest --collect-only` in all the different output formats from v3.2.5 to v6.2.4 and use it consistently for `spack unit-test --list-*` - Updating the unit tests in Github Actions to use a more recent `pytest` version
2021-11-18ci: run style unit tests only if we target develop (#27472)Harmen Stoppels2-2/+17
Some tests assume the base branch is develop, but this branch may not have been checked out.
2021-11-17Fix overly generic exceptions in log parser (#27413)Harmen Stoppels1-4/+0
This type of error is skipped: make[1]: *** [Makefile:222: /tmp/user/spack-stage/.../spack-src/usr/lib/julia/libopenblas64_.so.so] Error 1 but it's useful to have it, especially when a package sets a variable incorrectly in makefiles
2021-11-16Intel mpi: allow use of external libfabric (#27292)Robert Cohn1-9/+10
Intel mpi comes with an installation of libfabric (which it needs as a dependency). It can use other implementations of libfabric at runtime though, so if you install a package that depends on `mpi` and `libfabric`, you can specify `intel-mpi+external-libfabric` and ensure that the Spack-built instance is used (both by `intel-mpi` and the root). Apply analogous change to intel-oneapi-mpi.
2021-11-15Turn some verbose messages into debug messages (#27408)Harmen Stoppels1-8/+8
2021-11-12Add PyPI docs and warning in auto-generated package (#27404)Seth R. Johnson2-3/+16
* docs: Add cross-references for pypi setup * create: add warning for missing pypi
2021-11-11Fix overloaded argparse keys (#27379)Harmen Stoppels5-15/+14
Commands should not reuse option names defined in main.
2021-11-09spack tutorial: fix output to screen (#27316)Massimiliano Culpo1-1/+3
Spack was checking out v0.17, the output reported v0.16
2021-11-09Fix typos (ouptut) (#27317)Maxim Belkin2-2/+2
2021-11-09Fix log-format reporter ignoring install errors (#25961)Jordan Galby3-31/+48
When running `spack install --log-format junit|cdash ...`, install errors were ignored. This made spack continue building dependents of failed install, ignoring `--fail-fast`, and exit 0 at the end.
2021-11-09make --enable-locks actually enable locks (#24675)Dylan Simon1-2/+3
2021-11-09build_environment: clean *_ROOT variables (#26474)Valentin Volkl1-0/+7
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-11-08Python tests: skip importing weirdly-named modules (#27151)iarspider2-0/+6
* Python tests: allow importing weirdly-named modules e.g. with dashes in name * SIP tests: allow importing weirdly-named modules * Skip modules with invalid names * Changes from review * Update from review * Update from review * Cleanup
2021-11-05bump version number to 0.17.0Todd Gamblin2-2/+2
2021-11-05Prevent additional properties to be in the answer set when reusing specs ↵Massimiliano Culpo3-1/+42
(#27240) * Prevent additional properties to be in the answer set when reusing specs fixes #27237 The mechanism to reuse concrete specs relies on imposing the set of constraints stemming from the concrete spec being reused. We also need to prevent that other constraints get added to this set.
2021-11-05make version docs reflect reality (#27149)Harmen Stoppels1-26/+30
* make version docs reflect reality * typo and make things * 2.6 -> 2.7 in example
2021-11-05commands: `spack load --list` alias for `spack find --loaded` (#27184)Todd Gamblin4-10/+50
See #25249 and https://github.com/spack/spack/pull/27159#issuecomment-958163679. This adds `spack load --list` as an alias for `spack find --loaded`. The new command is not as powerful as `spack find --loaded`, as you can't combine it with all the queries or formats that `spack find` provides. However, it is more intuitively located in the command structure in that it appears in the output of `spack load --help`. The idea here is that people can use `spack load --list` for simple stuff but fall back to `spack find --loaded` if they need more. - add help to `spack load --list` that references `spack find` - factor some parts of `spack find` out to be called from `spack load` - add shell tests - update docs Co-authored-by: Peter Josef Scheibel <scheibel1@llnl.gov> Co-authored-by: Richarda Butler <39577672+RikkiButler20@users.noreply.github.com>
2021-11-05docs for experimental `--reuse` argument to `spack install`Todd Gamblin1-0/+28
Add docs for `--reuse`, along with a warning that it will likely be removed and refactored.
2021-11-05error message for reusing multiple hashes for packageGregory Becker1-1/+2
2021-11-05concretizer: add error messages and simplify asp.pyTodd Gamblin2-8/+18
2021-11-05Fix logic program for multi-valued variantsMassimiliano Culpo1-45/+71
Reformulate variant rules so that we minimize both 1. The number of non-default values being used 2. The number of default values not-being used This is crucial for MV variants where we may have more than one default value
2021-11-05bugfix: handle hashes that only exist in input specsTodd Gamblin1-27/+48
In our tests, we use concrete specs generated from mock packages, which *only* occur as inputs to the solver. This fixes two problems: 1. We weren't previously adding facts to encode the necessary `depends_on()` relationships, and specs were unsatisfiable on reachability. 2. Our hash lookup for reconstructing the DAG does not consider that a hash may have come from the inputs.
2021-11-05concretizer: exempt already-installed specs from compiler and variant rulesTodd Gamblin2-7/+22
Concrete specs that are already installed or that come from a buildcache may have compilers and variant settings that we do not recognize, but that shouldn't prevent reuse (at least not until we have a more detailed compiler model). - [x] make sure compiler and variant consistency rules only apply to built specs - [x] don't validate concrete specs on input, either -- they're concrete and we shouldn't apply today's rules to yesterday's build
2021-11-05spack diff: more flexible tests, restore transitive diff with spec_clausesTodd Gamblin3-12/+39
In switching to hash facts for concrete specs, we lost the transitive facts from dependencies. This was fine for solves, because they were implied by the imposed constraints from every hash. However, for `spack diff`, we want to see what the hashes mean, so we need another mode for `spec_clauses()` to show that. This adds a `expand_hashes` argument to `spec_clauses()` that allows us to output *both* the hashes and their implications on dependencies. We use this mode in `spack diff`.
2021-11-05Add a missing definition in the logic programMassimiliano Culpo1-0/+1
2021-11-05Add buildcache to reusable specsMassimiliano Culpo4-16/+51
2021-11-05spack install: add --reuse argumentMassimiliano Culpo2-4/+6
2021-11-05spack concretize: add --reuse argumentMassimiliano Culpo4-16/+35
2021-11-05spack spec: add --reuse argumentMassimiliano Culpo4-16/+33
2021-11-05concretizer: get rid of last maximize directive in concretize.lpTodd Gamblin1-32/+5
- [x] Get rid of forgotten maximize directive. - [x] Simplify variant handling - [x] Fix bug in treatment of defaults on externals (don't count non-default variants on externals against them)
2021-11-05Trim dependencies on externalsMassimiliano Culpo1-1/+2
2021-11-05Fix a unit test to match the new OS semanticsMassimiliano Culpo1-1/+1
CNL, debian6 and Suse are not compatible
2021-11-05ASP-based solve: if an OS is set, respect the valueMassimiliano Culpo1-0/+3
2021-11-05Fix a type in "variant_not_default" ruleMassimiliano Culpo1-1/+1
2021-11-05concretizer: rework spack solve output to handle reuse betterTodd Gamblin3-75/+137
2021-11-05spec: ensure_valid_variants() should not validate concrete specsTodd Gamblin1-0/+4
Variants in concrete specs are "always" correct -- or at least we assume them to be b/c they were concretized before. Their variants need not match the current version of the package.
2021-11-05concretizer: unify handling of single- and multi-valued variantsTodd Gamblin1-18/+5
Multi-valued variants previously maximized default values to handle cases where the default contained two values, e.g.: variant("foo", default="bar,baz") This is because previously we were minimizing non-default values, and `foo=bar`, `foo=baz`, and `foo=bar,baz` all had the same score, as none of them had any "non-default" values. This commit changes the approach and considers a non-default value to be either a value set to something not default *or* the absence of a default value from the set value. This allows multi- and single-valued variants to be handled the same way, with the same minimization criterion. It alse means that the "best" value for every optimization criterion is now zero, which allows us to make useful assumptions about the optimization criteria.
2021-11-05concretizer: reuse installs, but assign default values for new buildsTodd Gamblin1-35/+119
Minimizing builds is tricky. We want a minimizing criterion because we want to reuse the avaialble installs, but we also want things that have to be built to stick to *default preferences* from the package and from the user. We therefore treat built specs differently and apply a different set of optimization criteria to them. Spack's *first* priority is to reuse what it can, but if it builds something, the built specs will respect defaults and preferences. This is implemented by bumping the priority of optimization criteria for built specs -- so that they take precedence over the otherwise topmost-priority criterion to reuse what is installed. The scheme relies on all of our optimization criteria being minimizations. That is, we need the case where all specs are reused to be better than any built spec could be. Basically, if nothing is built, all the build criteria are zero (the best possible) and the number of built packages dominates. If something *has* to be built, it must be strictly worse than full reuse, because: 1. it increases the number of built specs 2. it must have either zero or some positive number for all criteria Our optimziation criteria effectively sum into two buckets at once to accomplish this. We use a `build_priority()` number to shift the priority of optimization criteria for built specs higher.
2021-11-05tests: make `spack diff` test more lenientTodd Gamblin1-4/+12
The constraints in the `spack diff` test were very specific and assumed a lot about the structure of what was being diffed. Relax them a bit to make them more resilient to changes.
2021-11-05concretizer: only minimize builds when `--reuse` is enabled.Todd Gamblin2-1/+4
Make the first minimization conditional on whether `--reuse` is enabled in the solve. If `--reuse` is not enabled, there will be nothing in the set to minimize and the objective function (for this criterion) will be 0 for every answer set.