summaryrefslogtreecommitdiff
path: root/share
AgeCommit message (Collapse)AuthorFilesLines
2022-05-20errors: model error messages as an optimization problem (#30669)Greg Becker1-1/+1
Error messages for the clingo concretizer have proven challenging. The current messages are incredibly vague and often don't help users at all. Unsat cores in clingo are not guaranteed to be minimal, and lead to cores that are either not useful or need to be post-processed for hours to reach a minimal core. Following up on an idea from a slack conversation with kwryankrattiger on slack, this PR takes a new approach. We eliminate most integrity constraints and minima/maxima on choice rules in clingo, and instead force invalid states to imply an error predicate. The error predicate can include context on the cause of the error (Package, Version, etc). These error predicates are then heavily optimized against, to ensure that we do not include error facts in the solution when a solution with no error facts could be generated. When post-processing the clingo solution to construct specs, any error facts cause the program to raise an exception. This leads to much more legible error messages. Each error predicate includes a priority and an error message. The error message is formatted by the remaining arguments to produce the error message. The priority is used to ensure that when clingo has a choice of which rules to violate, it chooses the one which will be most informative to the user. Performance: "fresh" concretizations appear to suffer a ~20% performance penalty under this branch, while "reuse" concretizations see a speedup of around 33%. Possible optimizations if users still see unhelpful messages: There are currently 3 levels of priority of the error messages. Additional priorities are possible, and can allow us finer granularity to ensure more informative error messages are provided in lieu of less informative ones. Future work: Improve tests to ensure that every possible rule implying an error message is exercised
2022-05-14Update GitLab environment variable name (#30671)Zack Galbreath1-2/+2
Use the IAM credentials that correspond to our new binary mirror (s3://spack-binaries vs. s3://spack-binaries-develop)
2022-05-13Remove all uses of `runtime_hash`; document lockfile formats and fix testsTodd Gamblin1-1/+1
This removes all but one usage of runtime hash. The runtime hash was being used to write historical lockfiles for tests, but we don't need it for that; we can just save those lockfiles. - [x] add legacy lockfiles for v1, v2, v3 - [x] fix bugs with v1 lockfile tests (the dummy lockfile we were writing was not actually a v1 lockfile because it used the new spec file format). - [x] remove all but one runtime_hash usage -- that one needs a small rework of the concretizer to really fix, as it's about separate concretization of build dependencies. - [x] Document the history of the lockfile format in `environment/__init__.py`
2022-05-13gitlab ci: switch over to new bucket for all stacksScott Wittenburg7-7/+7
2022-05-13hashes: remove full_hash and build_hash from spackScott Wittenburg1-2/+2
2022-05-13Reuse concretization by default (#30396)Massimiliano Culpo7-0/+21
* Enable reuse by default in Spack * Update documentation to match new default * Configure pipelines not to reuse software
2022-05-13tutorial stack: allow deprecated versions (#30648)Todd Gamblin1-0/+3
For tutorial builds, we should continue to allow deprecated builds to be installed. We can update them as needed when we update the tutorial, but we don't need to correct them immediately on deprecation in CI. - [x] add `deprecated:true` to tutorial `spack.yaml` config.
2022-05-10gitlab ci: do not override .generate tags for e4s (#30571)Scott Wittenburg1-1/+0
2022-05-09e4s on mac ci: set SPACK_DISABLE_LOCAL_CONFIG=1 (#30568)eugeneswalker2-4/+43
* e4s on mac ci: set SPACK_DISABLE_LOCAL_CONFIG=1 * export SPACK_USER_CACHE_PATH so that ~/.spack/... isn't used
2022-05-06ci: Enable the ParaView GUI in the DAVSDK pipeline (#30473)Chuck Atkins1-5/+7
* ci: Enable the ParaView GUI in the DAVSDK pipeline * qt: Patch for long paths in ci
2022-05-05Makefile generator for parallel spack install of environments (#30254)Harmen Stoppels1-1/+5
`make` solves a lot of headaches that would otherwise have to be implemented in Spack: 1. Parallelism over packages through multiple `spack install` processes 2. Orderly output of parallel package installs thanks to `make --sync-output=recurse` or `make -Orecurse` (works well in GNU Make 4.3; macOS is unfortunately on a 16 years old 3.x version, but it's one `spack install gmake` away...) 3. Shared jobserver across packages, which means a single `-j` to rule them all, instead of manually finding a balance between `#spack install processes` & `#jobs per package` (See #30302). This pr adds the `spack env depfile` command that generates a Makefile with dag hashes as targets, and dag hashes of dependencies as prerequisites, and a command along the lines of `spack install --only=packages /hash` to just install a single package. It exposes two convenient phony targets: `all`, `fetch-all`. The former installs the environment, the latter just fetches all sources. So one can either use `make all -j16` directly or run `make fetch-all -j16` on a login node and `make all -j16` on a compute node. Example: ```yaml spack: specs: [perl] view: false ``` running ``` $ spack -e . env depfile --make-target-prefix env | tee Makefile ``` generates ```Makefile SPACK ?= spack .PHONY: env/all env/fetch-all env/clean env/all: env/env env/fetch-all: env/fetch env/env: env/.install/cdqldivylyxocqymwnfzmzc5sx2zwvww @touch $@ env/fetch: env/.fetch/cdqldivylyxocqymwnfzmzc5sx2zwvww env/.fetch/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.fetch/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.fetch/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.fetch/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.fetch/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.fetch/73t7ndb5w72hrat5hsax4caox2sgumzu env/.fetch/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.fetch/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/.fetch/c4go4gxlcznh5p5nklpjm644epuh3pzc @touch $@ env/dirs: @mkdir -p env/.fetch env/.install env/.fetch/%: | env/dirs $(info Fetching $(SPEC)) $(SPACK) -e '/tmp/tmp.7PHPSIRACv' fetch $(SPACK_FETCH_FLAGS) /$(notdir $@) && touch $@ env/.install/%: env/.fetch/% $(info Installing $(SPEC)) +$(SPACK) -e '/tmp/tmp.7PHPSIRACv' install $(SPACK_INSTALL_FLAGS) --only-concrete --only=package --no-add /$(notdir $@) && touch $@ # Set the human-readable spec for each target env/%/cdqldivylyxocqymwnfzmzc5sx2zwvww: SPEC = perl@5.34.1%gcc@10.3.0+cpanm+shared+threads arch=linux-ubuntu20.04-zen2 env/%/gv5kin2xnn33uxyfte6k4a3bynhmtxze: SPEC = berkeley-db@18.1.40%gcc@10.3.0+cxx~docs+stl patches=b231fcc arch=linux-ubuntu20.04-zen2 env/%/cuymc7e5gupwyu7vza5d4vrbuslk277p: SPEC = bzip2@1.0.8%gcc@10.3.0~debug~pic+shared arch=linux-ubuntu20.04-zen2 env/%/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk: SPEC = diffutils@3.8%gcc@10.3.0 arch=linux-ubuntu20.04-zen2 env/%/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws: SPEC = libiconv@1.16%gcc@10.3.0 libs=shared,static arch=linux-ubuntu20.04-zen2 env/%/yfz2agazed7ohevqvnrmm7jfkmsgwjao: SPEC = gdbm@1.19%gcc@10.3.0 arch=linux-ubuntu20.04-zen2 env/%/73t7ndb5w72hrat5hsax4caox2sgumzu: SPEC = readline@8.1%gcc@10.3.0 arch=linux-ubuntu20.04-zen2 env/%/trvdyncxzfozxofpm3cwgq4vecpxixzs: SPEC = ncurses@6.2%gcc@10.3.0~symlinks+termlib abi=none arch=linux-ubuntu20.04-zen2 env/%/sbzszb7v557ohyd6c2ekirx2t3ctxfxp: SPEC = pkgconf@1.8.0%gcc@10.3.0 arch=linux-ubuntu20.04-zen2 env/%/c4go4gxlcznh5p5nklpjm644epuh3pzc: SPEC = zlib@1.2.12%gcc@10.3.0+optimize+pic+shared patches=0d38234 arch=linux-ubuntu20.04-zen2 # Install dependencies env/.install/cdqldivylyxocqymwnfzmzc5sx2zwvww: env/.install/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.install/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.install/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.install/c4go4gxlcznh5p5nklpjm644epuh3pzc env/.install/cuymc7e5gupwyu7vza5d4vrbuslk277p: env/.install/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.install/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk: env/.install/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.install/yfz2agazed7ohevqvnrmm7jfkmsgwjao: env/.install/73t7ndb5w72hrat5hsax4caox2sgumzu env/.install/73t7ndb5w72hrat5hsax4caox2sgumzu: env/.install/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.install/trvdyncxzfozxofpm3cwgq4vecpxixzs: env/.install/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/clean: rm -f -- env/env env/fetch env/.fetch/cdqldivylyxocqymwnfzmzc5sx2zwvww env/.fetch/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.fetch/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.fetch/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.fetch/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.fetch/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.fetch/73t7ndb5w72hrat5hsax4caox2sgumzu env/.fetch/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.fetch/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/.fetch/c4go4gxlcznh5p5nklpjm644epuh3pzc env/.install/cdqldivylyxocqymwnfzmzc5sx2zwvww env/.install/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.install/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.install/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.install/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.install/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.install/73t7ndb5w72hrat5hsax4caox2sgumzu env/.install/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.install/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/.install/c4go4gxlcznh5p5nklpjm644epuh3pzc ``` Then with `make -O` you get very nice orderly output when packages are built in parallel: ```console $ make -Orecurse -j16 spack -e . install --only-concrete --only=package /c4go4gxlcznh5p5nklpjm644epuh3pzc && touch c4go4gxlcznh5p5nklpjm644epuh3pzc ==> Installing zlib-1.2.12-c4go4gxlcznh5p5nklpjm644epuh3pzc ... Fetch: 0.00s. Build: 0.88s. Total: 0.88s. [+] /tmp/tmp.b1eTyAOe85/store/linux-ubuntu20.04-zen2/gcc-10.3.0/zlib-1.2.12-c4go4gxlcznh5p5nklpjm644epuh3pzc spack -e . install --only-concrete --only=package /sbzszb7v557ohyd6c2ekirx2t3ctxfxp && touch sbzszb7v557ohyd6c2ekirx2t3ctxfxp ==> Installing pkgconf-1.8.0-sbzszb7v557ohyd6c2ekirx2t3ctxfxp ... Fetch: 0.00s. Build: 3.96s. Total: 3.96s. [+] /tmp/tmp.b1eTyAOe85/store/linux-ubuntu20.04-zen2/gcc-10.3.0/pkgconf-1.8.0-sbzszb7v557ohyd6c2ekirx2t3ctxfxp ``` For Perl, at least for me, using `make -j16` versus `spack -e . install -j16` speeds up the builds from 3m32.623s to 2m22.775s, as some configure scripts run in parallel. Another nice feature is you can do Makefile "metaprogramming" and depend on packages built by Spack. This example fetches all sources (in parallel) first, print a message, and only then build packages (in parallel). ```Makefile SPACK ?= spack .PHONY: env all: env spack.lock: spack.yaml $(SPACK) -e . concretize -f env.mk: spack.lock $(SPACK) -e . env depfile -o $@ --make-target-prefix spack fetch: spack/fetch @echo Fetched all packages && touch $@ env: fetch spack/env @echo This executes after the environment has been installed clean: rm -rf spack/ env.mk spack.lock ifeq (,$(filter clean,$(MAKECMDGOALS))) include env.mk endif ```
2022-05-05spack external find: add search path customization (#30479)Greg Becker1-1/+1
2022-05-04gitlab: Remove temporary storage url from all stacks (#29949)Scott Wittenburg7-10/+2
Gitlab pipelines run for spack already have other S3 storage locations configured for storage of binaries, so this PR removes the redundant per-pipeline mirror. As a result, the "cleanup" jobs will no longer be generated at the end of each pipeline, removing one possible point of pipeline failure.
2022-05-04Remove deprecated "--run-tests" option of "spack install" (#30461)Massimiliano Culpo1-1/+1
2022-04-28Delocalize type output for bash completion (#30360)lpoirel2-2/+2
2022-04-28Add command for reading JSON-based DB description (now with more tests) (#29652)Peter Scheibel1-2/+6
This is an amended version of https://github.com/spack/spack/pull/24894 (reverted in https://github.com/spack/spack/pull/29603). https://github.com/spack/spack/pull/24894 broke all instances of `spack external find` (namely when it is invoked without arguments/options) because it was mandating the presence of a file which most systems would not have. This allows `spack external find` to proceed if that file is not present and adds tests for this. - [x] Add a test which confirms that `spack external find` successfully reads a manifest file if present in the default manifest path --- Original commit message --- Adds `spack external read-cray-manifest`, which reads a json file that describes a set of package DAGs. The parsed results are stored directly in the database. A user can see these installed specs with `spack find` (like any installed spec). The easiest way to use them right now as dependencies is to run `spack spec ... ^/hash-of-external-package`. Changes include: * `spack external read-cray-manifest --file <path/to/file>` will add all specs described in the file to Spack's installation DB and will also install described compilers to the compilers configuration (the expected format of the file is described in this PR as well including examples of the file) * Database records now may include an "origin" (the command added in this PR registers the origin as "external-db"). In the future, it is assumed users may want to be able to treat installs registered with this command differently (e.g. they may want to uninstall all specs added with this command) * Hash properties are now always preserved when copying specs if the source spec is concrete * I don't think the hashes of installed-and-concrete specs should change and this was the easiest way to handle that * also specs that are concrete preserve their `.normal` property when copied (external specs may mention compilers that are not registered, and without this change they would fail in `normalize` when calling `validate_or_raise`) * it might be this should only be the case if the spec was installed - [x] Improve testing - [x] Specifically mark DB records added with this command (so that users can do something like "uninstall all packages added with `spack read-external-db`) * This is now possible with `spack uninstall --all --origin=external-db` (this will remove all specs added from manifest files) - [x] Strip variants that are listed in json entries but don't actually exist for the package
2022-04-28build_env/test_env: add concretizer args (#30289)Greg Becker1-2/+2
2022-04-27e4s ci: uncomment umpire (#29776)eugeneswalker1-1/+1
2022-04-26spack spec: add '--format' argument (#27908)lorddavidiii1-1/+1
2022-04-25Environments: add flag to skip printing concretized specs (#30272)iarspider1-1/+1
With an active environment, you can now run "spack concretize --quiet" and it will suppress printing the concretized specs.
2022-04-22Update Dockerfiles and images for Spack v0.18.0 (#30216)Massimiliano Culpo13-411/+46
This PR updates the list of images we build nightly, deprecating Ubuntu 16.04 and CentOS 8 and adding Ubuntu 20.04, Ubuntu 22.04 and CentOS Stream. It also removes a lot of duplication by generating the Dockerfiles during the CI workflow and uploading them as artifacts for later inspection or reuse.
2022-04-20refactor powershell setup to make it sourceable (#29987)Tom Scogland1-0/+61
* refactor powershell setup to make it sourceable * only set editor if it is unset * change directory to spack root in subshell * Update share/spack/setup-env.ps1 Co-authored-by: John W. Parent <45471568+johnwparent@users.noreply.github.com> Co-authored-by: John W. Parent <45471568+johnwparent@users.noreply.github.com>
2022-04-19Set resource requests on package builds (#29922)Christopher Kotfila5-46/+465
gitlab ci: Set resource requests explicitly This PR sets resource requests for the Kubernetes executor, which should aid in better workload scheduling in the cluster. The specific values were derived from profile data taken from several full "from scratch" rebuilds in a separate worker pool. Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com>
2022-04-14spack ci: remove relate-CDash-builds functionality (#29950)Zack Galbreath1-1/+1
gitlab ci: Remove code for relating CDash builds Relating CDash builds to their dependencies was a seldom used feature. Removing it will make it easier for us to reorganize our CDash projects & build groups in the future by eliminating the needs to keep track of CDash build ids in our binary mirrors.
2022-04-13Add support for Python 3.10 (#29581)Massimiliano Culpo4-0/+472
* Add support for Python 3.10 * Update unit-tests to use 3.10 * Update Getting started section of the docs * Update bootstrap action
2022-04-07Use the non-deprecated `MetaPathFinder` interface (#29745)Massimiliano Culpo1-0/+3
* Extract the MetaPathFinder and Loaders for packages in their own classes https://peps.python.org/pep-0451/ Currently, RepoPath and Repo implement the (deprecated) interface of MetaPathFinder (find_module) and of Loader (load_module). This commit extracts both of them and places the code in their own classes. The MetaPathFinder interface is updated to contain both the deprecated "find_module" (for Python 2.7 support) and the recommended "find_spec". Update of the Loader interface is deferred at a subsequent commit. * Move the lines to be prepended inside "RepoLoader" Also adjust the naming of a few variables too * Remove spack.util.imp, since code is only used in spack.repo * Remove support from loading Python modules Python > 3 but < 3.5 * Remove `Repo._create_namespace` This function was interacting badly with the MetaPathFinder and causing issues with "normal" imports. Removing the function allows to do things like: ```python import spack.pkg.builtin.mpich cls = spack.pkg.builtin.mpich.Mpich ``` * Remove code needed to trigger the Singleton evaluation The finder is coded in a way to trigger the Singleton, so we don't need external code now that we register it at module level into `sys.meta_path`. * Add unit tests
2022-04-07e4s ci: expand mac mini stack (#29929)eugeneswalker1-2/+2
2022-03-30spack ci: filter untouched pkgs from PR pipelines (#29697)Scott Wittenburg1-0/+1
We've previously generated CI pipelines for PRs, and they rebuild any packages that don't have a binary in an existing build cache. The assumption we were making was that ALL prior merged builds would be in cache, but due to the way we do security in the pipeline, they aren't. `develop` pipelines can take a while to catch up with the latest PRs, and while it does that, there may be a bunch of redundant builds on PRs that duplicate things being rebuilt on `develop`. Until we can do better caching of PR builds, we'll have this problem. We can do better in PRs, though, by *only* rebuilding things in the CI environment that are actually touched by the PR. This change computes exactly what packages are changed by a PR branch and *only* includes those packages' dependents and dependencies in the generated pipeline. Other as-yet unbuilt packages are pruned from CI for the PR. For `develop` pipelines, we still want to build everything to ensure that the stack works, and to ensure that `develop` catches up with PRs. This is especially true since we do not do rebuilds for *every* commit on `develop` -- just the most recent one after each `develop` pipeline finishes. Since we skip around, we may end up missing builds unless we ensure that we rebuild everything. We differentiate between `develop` and PR pipelines in `.gitlab-ci.yml` by setting `SPACK_PRUNE_UNTOUCHED` for PRs. `develop` will still have the old behavior. - [x] Add `SPACK_PRUNE_UNTOUCHED` variable to `spack ci` - [x] Refactor `spack pkg` command by moving historical package checking logic to `spack.repo` - [x] Implement pruning logic in `spack ci` to remove untouched packages - [x] add tests
2022-03-28spack info: make sections optional; add build/stand-alone test information ↵Tamara Dahlgren1-1/+1
(#22097) Add output of build- and install-time tests to info command Enable dependencies, variants, and versions by default (i.e., provide --no* options; add gcc to test_info_fields to increase coverage for c_names->v_names
2022-03-28e4s ci spack.yaml: add h5bench (#29755)eugeneswalker1-0/+1
2022-03-21e4s mini mac stack: add bzip2 (#29650)eugeneswalker1-0/+1
2022-03-19Revert "Add command for reading a json-based DB description (#24894)" (#29603)Nils Vu1-6/+2
This reverts commit 531b1c5c3dcc9bc7bec27e223608aed477e94dbd.
2022-03-18Add command for reading a json-based DB description (#24894)Peter Scheibel1-2/+6
Adds `spack external read-cray-manifest`, which reads a json file that describes a set of package DAGs. The parsed results are stored directly in the database. A user can see these installed specs with `spack find` (like any installed spec). The easiest way to use them right now as dependencies is to run `spack spec ... ^/hash-of-external-package`. Changes include: * `spack external read-cray-manifest --file <path/to/file>` will add all specs described in the file to Spack's installation DB and will also install described compilers to the compilers configuration (the expected format of the file is described in this PR as well including examples of the file) * Database records now may include an "origin" (the command added in this PR registers the origin as "external-db"). In the future, it is assumed users may want to be able to treat installs registered with this command differently (e.g. they may want to uninstall all specs added with this command) * Hash properties are now always preserved when copying specs if the source spec is concrete * I don't think the hashes of installed-and-concrete specs should change and this was the easiest way to handle that * also specs that are concrete preserve their `.normal` property when copied (external specs may mention compilers that are not registered, and without this change they would fail in `normalize` when calling `validate_or_raise`) * it might be this should only be the case if the spec was installed - [x] Improve testing - [x] Specifically mark DB records added with this command (so that users can do something like "uninstall all packages added with `spack read-external-db`) * This is now possible with `spack uninstall --all --origin=external-db` (this will remove all specs added from manifest files) - [x] Strip variants that are listed in json entries but don't actually exist for the package Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-03-17Windows Support: Testing Suite integrationJohn Parent1-1/+1
Broaden support for execution of the test suite on Windows. General bug and review fixups
2022-03-17"spack commands --update-completion"John Parent1-1/+1
2022-03-17Add Github Actions for Windows (#24504)John Parent5-5/+41
Setup Installer CI (#25184), (#25191) Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com> Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17Windows: Create installer and environmentlou.lawrence@kitware.com1-1/+10
* Add 'make-installer' command for Windows * Add '--bat' arg to env activate, env deactivate and unload commands * An equivalent script to setup-env on linux: spack_cmd.bat. This script has a wrapper to evaluate cd, load/unload, env activate/deactivate.(#21734) * Add spacktivate and config editor (#22049) * spack_cmd: will find python and spack on its own. It preferentially tries to use python on your PATH (#22414) * Ignore Windows python installer if found (#23134) * Bundle git in windows installer (#23597) * Add Windows section to Getting Started document (#23131), (#23295), (#24240) Co-authored-by: Stephen Crowell <stephen.crowell@kitware.com> Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com> Co-authored-by: Jared Popelar <jpopelar@txcorp.com> Co-authored-by: Ben Cowan <benc@txcorp.com> Update Installer CI Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-14ci: add e4s mac stack (#29476)eugeneswalker2-0/+105
2022-03-09Fix tab completion erroring with `spack unit-test` (#29405)百地 希留耶2-5/+5
2022-03-08Fix overconstrained HDF5 variants (#29132)Seth R. Johnson1-2/+0
* hdf5: mark +fortran+shared conflict for older version This version was only activated unintentionally by silo's conflict statement, but `@1.8.15+shared+fortran+cxx` errors out in configure: ``` CMake Error at CMakeLists.txt:814 (message): **** Shared FORTRAN libraries are unsupported **** ``` * silo: refine hdf5 conflicts to avoid building old version Before this, `silo+hdf5` concretized to 1.10.7 or sometimes 1.8.15. Now I've verified it works for the following configurations: ``` silo@4.10.2 patches=7b5a1dc,952d3c9 ^ hdf5@1.10.7 api=default silo@4.10.2 patches=7b5a1dc,952d3c9,eb2a3a0 ^ hdf5@1.10.8 api=v18 silo@4.10.2 patches=7b5a1dc,952d3c9,eb2a3a0 ^ hdf5@1.12.1 api=v110 silo@4.11-bsd patches=eb2a3a0 ^ hdf5@1.12.1 api=v110 silo@4.11-bsd patches=eb2a3a0 ^ hdf5@1.10.8 api=default silo@4.11-bsd patches=eb2a3a0 ^ hdf5@1.12.1 api=default ``` and verified that the following fail: ``` silo@4.10.2 ^hdf5@1.12.1 api=default silo@4.11 ^hdf5 api=v18 silo@4.11-bsd ^hdf5@1.13.0 api=v12 silo@4.11-bsd ^hdf5@1.13.0 api=default ``` and have updated the constraints to match. Hdf5 no longer has to be downgraded to work with Silo. * silo: fix dependency conflicts * py-h5py: shorten and add comments to py-h5py hdf5 dependencies * e4s: remove slightly outdated hdf5 requirement * e4s: remove excessive hdf5 variant constraints These I think are holdovers from the old concretizer. - `hdf5_compat` can be expressed as `+hdf5 ^hdf5@1.8` - The extra variants on hdf5 shouldn't break conduit - axom unnecessarily restricts hdf5 version * conduit: restore hdf5_compat flag
2022-02-28Add a new test to catch exit code failure (#29244)Massimiliano Culpo1-0/+3
* Add a new test to catch exit code failure fixes #29226 This introduces a new unit test that checks the return code of `spack unit-test` when it is supposed to fail. This is to prevent bugs like the one introduced in #25601 in which CI didn't catch a missing return statement. In retrospective it seems that the shell test we have right now all go through `tty.die` or similar code paths which call `sys.exit(a)` explicitly. This new test instead checks `spack unit-test` which relies on the return code from command invocation in case of errors.
2022-02-22e4s ci: packages: prefer openturns@1.18 (#29154)eugeneswalker1-0/+2
2022-02-22Add `spack --bootstrap` option for accessing bootstrap store (#25601)Todd Gamblin1-1/+1
We can see what is in the bootstrap store with `spack find -b`, and you can clean it with `spack clean -b`, but we can't do much else with it, and if there are bootstrap issues they can be hard to debug. We already have `spack --mock`, which allows you to swap in the mock packages from the command line. This PR introduces `spack -b` / `spack --bootstrap`, which runs all of spack with `ensure_bootstrap_configuration()` set. This means that you can run `spack -b find`, `spack -b install`, `spack -b spec`, etc. to see what *would* happen with bootstrap configuration, to remove specific bootstrap packages, etc. This will hopefully make developers' lives easier as they deal with bootstrap packages. This PR also uses a `nullcontext` context manager. `nullcontext` has been implemented in several other places in Spack, and this PR consolidates them to `llnl.util.lang`, with a note that we can delete the function if we ever reqyire a new enough Python. - [x] introduce `spack --bootstrap` option - [x] consolidated all `nullcontext` usages to `llnl.util.lang`
2022-02-18spack external find: change default behavior (#29031)Massimiliano Culpo1-1/+1
See https://github.com/spack/spack/issues/25353#issuecomment-1041868116 This commit changes the default behavior of ``` $ spack external find ``` from searching all the possible packages Spack knows about to search only for the ones tagged as being a "build-tool". It also introduces a `--all` option to restore the old behavior.
2022-02-17Testing: optionally run tests on externally installed packages (#28701)Tamara Dahlgren1-1/+1
Since Spack does not install external packages, this commit skips them by default when running stand-alone tests. The assumption is that such packages have likely undergone an acceptance test process. However, the tests can be run against installed externals using ``` % spack test run --externals ... ```
2022-02-16commands: refactor `--reuse` handling to use configTodd Gamblin1-5/+5
`--reuse` was previously handled individually by each command that needed it. We are growing more concretization options, and they'll need their own section for commands that support them. Now there are two concretization options: * `--reuse`: Attempt to reuse packages from installs and buildcaches. * `--fresh`: Opposite of reuse -- traditional spack install. To handle thes, this PR adds a `ConfigSetAction` for `argparse`, so that you can write argparse code like this: ``` subgroup.add_argument( '--reuse', action=ConfigSetAction, dest="concretizer:reuse", const=True, default=None, help='reuse installed dependencies/buildcaches when possible' ) ``` With this, you don't need to add logic to pull the argument out and handle it; the `ConfigSetAction` just does it for you. This can probably be used to clean up some other commands later, as well. Code that was previously passing `reuse=True` around everywhere has been refactored to use config, and config is set from the CLI using a new `add_concretizer_args()` function in `spack.cmd.common.arguments`. - [x] Add `ConfigSetAction` to simplify concretizer config on the CLI - [x] Refactor code so that it does not pass `reuse=True` to every function. - [x] Refactor commands to use `add_concretizer_args()` and to pass concretizer config using the config system.
2022-02-10Stabilize the concretization of ecp-data-vis-sdk by preferring conduit@0.7.2 ↵Massimiliano Culpo1-0/+3
(#28871)
2022-02-10e4s: new specs: nccmp, nco, wannier90, lammps (#28833)eugeneswalker1-0/+4
2022-02-04e4s ci: uncomment variorum following pr #28754 (#28763)eugeneswalker2-2/+2
2022-02-02Revert "Deprecate Python 2 installations" (#28411)Adam J. Stewart1-0/+1
This reverts commit 7b76e3982f94ee952fe5d8fb0d19b389cc28228a.