summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2022-05-13directory_layout: remove outdated checks for old DAG hashTodd Gamblin2-34/+6
We previously had checks in `directory_layout` to check for build-dependency conflicts when we weren't storing build dependencies. We don't need those anymore; we can just rely on the DAG hash now that it includes everything we know about each spec. - [x] Remove vestigial code for checking installed spec against concrete spec in `ensure_installed()` - [x] Remove `SpecHashCollisionError` -- if specs have the same hash now, they're the same as far as `DirectoryLayout` should be concerned. - [x] Convert spec comparison to `dag_hash()` comparison when adding extensions.
2022-05-13full hash: fix uninstall and gc with full hash DBTodd Gamblin3-5/+26
The database now stores full hashes, so we need to adjust the criteria we use to determine if something can be uninstalled. Specifically, it's ok to uninstall thing that have remaining build-only dependents.
2022-05-13concretizer: enable hash reuse with full hashTodd Gamblin7-87/+156
With the original DAG hash, we did not store build dependencies in the database, but with the full DAG hash, we do. Previously, we'd never tell the concretizer about build dependencies of things used by hash, because we never had them. Now, we have to avoid telling the concretizer about them, or they'll unnecessarily constrain build dependencies for new concretizations. - [x] Make database track all dependencies included in the `dag_hash` - [x] Modify spec_clauses so that build dependency information is optional and off by default. - [x] `spack diff` asks `spec_clauses` for build dependencies for completeness - [x] Modify `concretize.lp` so that reuse optimization doesn't affect fresh installations. - [x] Modify concretizer setup so that it does *not* prioritize installed versions over package versions. We don't need this with reuse, so they're low priority. - [x] Fix `test_installed_deps` for full hash and new concretizer (does not work for old concretizer with full hash -- leave this for later if we need it) - [x] Move `test_installed_deps` mock packages to `builtin.mock` for easier debugging with `spack -m`. - [x] Fix `test_reuse_installed_packages_when_package_def_changes` for full hash
2022-05-13bugfix: tests trying to ignore package changes should use `build_hash`Todd Gamblin3-5/+15
- [x] update test to use `build_hash` instead of `dag_hash`, as we're testing for graph structure, and specifically NOT testing for package changes. - [x] make hash descriptors callable on specs to simplify syntax for invoking them - [x] make `Spec.spec_hash()` public
2022-05-13Remove all uses of `runtime_hash`; document lockfile formats and fix testsTodd Gamblin14-841/+2719
This removes all but one usage of runtime hash. The runtime hash was being used to write historical lockfiles for tests, but we don't need it for that; we can just save those lockfiles. - [x] add legacy lockfiles for v1, v2, v3 - [x] fix bugs with v1 lockfile tests (the dummy lockfile we were writing was not actually a v1 lockfile because it used the new spec file format). - [x] remove all but one runtime_hash usage -- that one needs a small rework of the concretizer to really fix, as it's about separate concretization of build dependencies. - [x] Document the history of the lockfile format in `environment/__init__.py`
2022-05-13`content_hash()`: make it work on abstract specsTodd Gamblin4-40/+78
Some test cases had to be modified in a kludgy way so that abstract specs made concrete would have versions on them. We shouldn't *need* to do this, as the only reason we care is because the content hash has to be able to get an archive for a version. This modifies the content hash so that it can be called on abstract specs, including only relevant content. This does NOT add a partial content hash to the DAG hash, as we do not really want that -- we don't need in-memory spec hashes to need to load package files. It just makes `Package.content_hash()` less prickly and tests easier to understand.
2022-05-13spec: fix serialization, avoid double call to node_dict_with_hashesTodd Gamblin1-1/+4
2022-05-13hashes: revert `spack monitor` hash changes to preserve original protocolTodd Gamblin4-14/+17
`spack monitor` expects a field called `spec_full_hash`, so we shouldn't change that. Instead, we can pass a `dag_hash` (which is now the full hash) but not change the field name.
2022-05-13fix full hash calls in `spack graph`Todd Gamblin1-4/+4
2022-05-13remove no longer needed full hash checkTodd Gamblin1-6/+1
2022-05-13spec: remove hashes_final as it's no longer needed.Todd Gamblin1-29/+1
`hashes_final` was used to indicate when a spec was concrete but possibly lacked `full_hash` or `build_hash` fields. This was only necessary because older Spacks didn't generate them, and we want to avoid recomputing them, as we likely do not have the same package files as existed at concretization time. Now, we don't need to do that -- there is only the DAG hash and specs are either concrete and have a `dag_hash`, or not concrete and have no `dag_hash`. There's no middle ground.
2022-05-13gitlab ci: Docstring methods or make them privateScott Wittenburg3-76/+238
2022-05-13binary_distribution: Refactor index generation into smaller methodsScott Wittenburg1-47/+57
2022-05-13env: Use order of roots to resolve DAG hash conflicts in legacy lockfilesScott Wittenburg2-26/+39
2022-05-13env: enforce predictable ordering when reading lockfileScott Wittenburg2-9/+19
Without some enforcement of spec ordering, python 2 produced different results in the affected test than did python 3. This change makes the arbitrary but reproducible decision to sort the specs by their lockfile key alphabetically.
2022-05-13spec: fix infinite recursion when computing package hashScott Wittenburg1-6/+14
Issue described in the following PR comment: https://github.com/spack/spack/pull/28504#issuecomment-1051835568 Solution described in subsequent comment: https://github.com/spack/spack/pull/28504#issuecomment-1053986132
2022-05-13Fix how environments are read from lockfileScott Wittenburg15-109/+832
2022-05-13hashes: remove full_hash and build_hash from spackScott Wittenburg27-710/+334
2022-05-13environment: key by dag_hash instead of build_hashScott Wittenburg5-45/+68
2022-05-13tests: fix failing test_hash_changeScott Wittenburg1-2/+4
The full hash appears twice in the spec dict now, replacing just the value replaces it under "hash" and "full_hash". Only replace the one that appears after "full_hash". I'm actually not sure what purpose this test served, so maybe it could be removed, as it may be testing some distinction between full and dag hash which no longer exists.
2022-05-13Include all deps and package content in the `dag_hash()`Todd Gamblin4-41/+45
For a long time, Spack has used a coarser hash to identify packages than it likely should. Packages are identified by `dag_hash()`, which includes only link and run dependencies. Build dependencies are stripped before hashing, and we have notincluded hashes of build artifacts or the `package.py` files used to build. This means the DAG hash actually doesn't represent all the things Spack can build, and it reduces reproducibility. We did this because, in the early days, users were (rightly) annoyed when a new version of CMake, autotools, or some other build dependency would necessitate a rebuild of their entire stack. Coarsening the hash avoided this issue and enabled a modicum of stability when only reusing packages by hash match. Now that we have `--reuse`, we don't need to be so careful. Users can avoid unnecessary rebuilds much more easily, and we can add more provenance to the spec without worrying that frequent hash changes will cause too many rebuilds. This commit starts the refactor with the following major change: - [x] Make `Spec.dag_hash()` include build, run, and link dependencides and the package hash (it is now equivalent to `full_hash()`). It also adds a couple of bugfixes for problems discovered during the switch: - [x] Don't add a `package_hash()` in `to_node_dict()` unless the spec is concrete (fixes breaks on abstract specs) - [x] Don't add source ids to the package hash for packages without a known fetch strategy (may mock packages are like this) - [x] Change how `Spec.patches` is memoized. Using `llnl.util.lang.memoized` on `Spec` objects causes specs to be stored in a `dict`, which means they need a hash. But, `dag_hash()` now includes patch `sha256`'s via the package hash, which can lead to infinite recursion
2022-05-13Reuse concretization by default (#30396)Massimiliano Culpo2-25/+20
* Enable reuse by default in Spack * Update documentation to match new default * Configure pipelines not to reuse software
2022-05-12Gitlab pipelines: add a small legend in the logs to interpret "[x]" (#30643)Massimiliano Culpo1-5/+2
2022-05-12Add cuda 11.7 compat bounds for gcc/clang (#30639)Harmen Stoppels1-2/+2
2022-05-11Allow read-only access to file cache (when needed) (#29693)Tamara Dahlgren2-2/+47
* Allow read-only access to file cache (when needed) * Tweaked and added unit tests * Skip test_cache_init_entry_fails for windows
2022-05-11Fix default buildcache location (#30230)Thomas Dickerson1-1/+1
Resolve path/URL parsing issues introduced by #27021
2022-05-10oneapi: add v2022.2 (#30531)Robert Cohn1-0/+9
2022-05-10bugfix: `spack pkg list` should be more picky about what's a package (#30577)Todd Gamblin1-3/+11
`spack pkg list` tests were broken by #29593 for cases when your `builtin.mock` repo still has stale backup files (or, really, stale directories) sitting around. This happens if you switch branches a lot. In this case, things like this were causing erroneous packages in the mock listing: ``` var/spack/repos/builtin.mock/packages/ foo/ package.py~ ``` - [x] make `list_packages` consider only directories with one-deep `package.py` files.
2022-05-10Add a Lua build-system (#28854)Tom Scogland9-27/+269
Reworking lua to allow easier substitution of the base lua implementation. Also adding in a maintained version of luajit and re-factoring the entire stack to use a custom build-system to centralize functionality like environment variable management and luarocks installation. The `lua-lang` virtual is now versioned so that a package that requires Lua 5.1 semantics can get any lua, but one that requires 5.2 will only get upstream lua. The luaposix package requires lua-bit32, but only when built with a lua conforming to version 5.1. This adds the package, and the dependencies, but exposed a problem with luarocks dependency detection. Since we're installing each package in its own "tree" and there's no environment variable to list extra trees, spack now generates a luarocks config file that lists all the trees of all the dependencies, and references it by setting `LUAROCKS_CONFIG` in the build environment of every LuaPackage. This allows luarocks to find the spack installed dependencies correctly rather than trying (and failing) to download them. Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Tom Scogland <tscogland@llnl.gov> Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-05-09tests: fix references to hard-coded `master` branch in `git` tests (#30572)Todd Gamblin2-9/+13
Some of our `git` tests still fail when `init.defaultBranch` is set to something other than `master`. - [x] get rid of all hard-coded `master` refs - [x] Use `'default'` to key tests that use the default branch
2022-05-09Get timeout for web requests with urllib from spack config, same as for curl ↵Dom Heinzeller1-5/+5
(#30468)
2022-05-09Windows permissions: uninstalling and cleaning stages (#29714)John W. Parent3-20/+74
When running on Windows, Spack may generate files in the stage/install prefixes that do not have write permissions, which prevents the removal of those directories (e.g. when cleaning stages or uninstalling). There should be a refactoring to avoid this in the first place, but that is assumed to be longer term, so the temporary fix is to make such files writable if they are not. This PR: * Automatically handles these permissions errors when uninstalling packages from the Spack root (makes then writable) * Updates similar already-existing logic when removing Spack-managed stage directories (the error-handling was assuming all errors were permissions errors and was therefore handling other errors inappropriately) Note: these permissions issues only appear on Windows so this logic is only applied there (permissions are not modified for this purpose on Linux etc.). This also adds special handling for a case where calling `isdir` on an `os.DirEntry` object would fail for improperly-created symlinks (e.g. on Windows, using `os.symlink` without `target_is_directory=True`). Note this specific issue only came up when enabling link_tree tests (specifically `source_merge_visitor_cant_be_cyclical`).
2022-05-08Cray manifest file: accept "nvidia" as "nvhpc" (#30428)Peter Scheibel2-13/+132
* create function for translating compiler names on specs/compiler entries in manifest * add tests for translating compiler names on spec/compiler entries * use higher-level function in test and add comment to prefer testing via higher-level function * opensuse clingo check should not fail on account of this pr, but I cannot get it to pass by restarting via CI UI
2022-05-07Force GCC to always provide a C++14 flag (#29781)Robert Pavel2-4/+2
* Force GCC to always provide a C++14 flag Updated gnu logic so that the c++14 flag for g++ is always propagated. This fixes issues with build systems that error out if passed an empty string for a flag. Engaging in the best kind of software engineering by updating the unit test to pass with the value it is now passed. This should better match the expected flag for g++ compiling with the C++14 standard
2022-05-06Fix improper type for InvalidDependencyError argument (#30504)Greg Becker2-3/+5
2022-05-06docs: jobserver & generated makefiles (#30526)Harmen Stoppels1-6/+8
2022-05-06Preserve jobserver file descriptors into build environment (#30302)Tom Scogland2-10/+29
This ensures that multiple spack instances called from `make` will respect the maximum number of jobs in the POSIX jobserver across packages. Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-05-05harden unit tests, enable basic parallelism (#29593)Tom Scogland7-23/+39
* use the init.defaultBranch name, not master * make tcl and modules/common independent Both used to use not just the same directory, but the same *file* for their outputs. In parallel this can cause problems, but it can also accidentally allow expected failures to pass if the file is left around by mistake. * use a non-global misc_cache in tests * make pkg tests resilient to gitignore * make source cache and module directories non-global
2022-05-05Makefile generator for parallel spack install of environments (#30254)Harmen Stoppels4-4/+317
`make` solves a lot of headaches that would otherwise have to be implemented in Spack: 1. Parallelism over packages through multiple `spack install` processes 2. Orderly output of parallel package installs thanks to `make --sync-output=recurse` or `make -Orecurse` (works well in GNU Make 4.3; macOS is unfortunately on a 16 years old 3.x version, but it's one `spack install gmake` away...) 3. Shared jobserver across packages, which means a single `-j` to rule them all, instead of manually finding a balance between `#spack install processes` & `#jobs per package` (See #30302). This pr adds the `spack env depfile` command that generates a Makefile with dag hashes as targets, and dag hashes of dependencies as prerequisites, and a command along the lines of `spack install --only=packages /hash` to just install a single package. It exposes two convenient phony targets: `all`, `fetch-all`. The former installs the environment, the latter just fetches all sources. So one can either use `make all -j16` directly or run `make fetch-all -j16` on a login node and `make all -j16` on a compute node. Example: ```yaml spack: specs: [perl] view: false ``` running ``` $ spack -e . env depfile --make-target-prefix env | tee Makefile ``` generates ```Makefile SPACK ?= spack .PHONY: env/all env/fetch-all env/clean env/all: env/env env/fetch-all: env/fetch env/env: env/.install/cdqldivylyxocqymwnfzmzc5sx2zwvww @touch $@ env/fetch: env/.fetch/cdqldivylyxocqymwnfzmzc5sx2zwvww env/.fetch/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.fetch/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.fetch/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.fetch/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.fetch/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.fetch/73t7ndb5w72hrat5hsax4caox2sgumzu env/.fetch/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.fetch/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/.fetch/c4go4gxlcznh5p5nklpjm644epuh3pzc @touch $@ env/dirs: @mkdir -p env/.fetch env/.install env/.fetch/%: | env/dirs $(info Fetching $(SPEC)) $(SPACK) -e '/tmp/tmp.7PHPSIRACv' fetch $(SPACK_FETCH_FLAGS) /$(notdir $@) && touch $@ env/.install/%: env/.fetch/% $(info Installing $(SPEC)) +$(SPACK) -e '/tmp/tmp.7PHPSIRACv' install $(SPACK_INSTALL_FLAGS) --only-concrete --only=package --no-add /$(notdir $@) && touch $@ # Set the human-readable spec for each target env/%/cdqldivylyxocqymwnfzmzc5sx2zwvww: SPEC = perl@5.34.1%gcc@10.3.0+cpanm+shared+threads arch=linux-ubuntu20.04-zen2 env/%/gv5kin2xnn33uxyfte6k4a3bynhmtxze: SPEC = berkeley-db@18.1.40%gcc@10.3.0+cxx~docs+stl patches=b231fcc arch=linux-ubuntu20.04-zen2 env/%/cuymc7e5gupwyu7vza5d4vrbuslk277p: SPEC = bzip2@1.0.8%gcc@10.3.0~debug~pic+shared arch=linux-ubuntu20.04-zen2 env/%/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk: SPEC = diffutils@3.8%gcc@10.3.0 arch=linux-ubuntu20.04-zen2 env/%/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws: SPEC = libiconv@1.16%gcc@10.3.0 libs=shared,static arch=linux-ubuntu20.04-zen2 env/%/yfz2agazed7ohevqvnrmm7jfkmsgwjao: SPEC = gdbm@1.19%gcc@10.3.0 arch=linux-ubuntu20.04-zen2 env/%/73t7ndb5w72hrat5hsax4caox2sgumzu: SPEC = readline@8.1%gcc@10.3.0 arch=linux-ubuntu20.04-zen2 env/%/trvdyncxzfozxofpm3cwgq4vecpxixzs: SPEC = ncurses@6.2%gcc@10.3.0~symlinks+termlib abi=none arch=linux-ubuntu20.04-zen2 env/%/sbzszb7v557ohyd6c2ekirx2t3ctxfxp: SPEC = pkgconf@1.8.0%gcc@10.3.0 arch=linux-ubuntu20.04-zen2 env/%/c4go4gxlcznh5p5nklpjm644epuh3pzc: SPEC = zlib@1.2.12%gcc@10.3.0+optimize+pic+shared patches=0d38234 arch=linux-ubuntu20.04-zen2 # Install dependencies env/.install/cdqldivylyxocqymwnfzmzc5sx2zwvww: env/.install/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.install/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.install/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.install/c4go4gxlcznh5p5nklpjm644epuh3pzc env/.install/cuymc7e5gupwyu7vza5d4vrbuslk277p: env/.install/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.install/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk: env/.install/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.install/yfz2agazed7ohevqvnrmm7jfkmsgwjao: env/.install/73t7ndb5w72hrat5hsax4caox2sgumzu env/.install/73t7ndb5w72hrat5hsax4caox2sgumzu: env/.install/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.install/trvdyncxzfozxofpm3cwgq4vecpxixzs: env/.install/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/clean: rm -f -- env/env env/fetch env/.fetch/cdqldivylyxocqymwnfzmzc5sx2zwvww env/.fetch/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.fetch/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.fetch/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.fetch/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.fetch/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.fetch/73t7ndb5w72hrat5hsax4caox2sgumzu env/.fetch/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.fetch/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/.fetch/c4go4gxlcznh5p5nklpjm644epuh3pzc env/.install/cdqldivylyxocqymwnfzmzc5sx2zwvww env/.install/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.install/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.install/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.install/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.install/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.install/73t7ndb5w72hrat5hsax4caox2sgumzu env/.install/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.install/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/.install/c4go4gxlcznh5p5nklpjm644epuh3pzc ``` Then with `make -O` you get very nice orderly output when packages are built in parallel: ```console $ make -Orecurse -j16 spack -e . install --only-concrete --only=package /c4go4gxlcznh5p5nklpjm644epuh3pzc && touch c4go4gxlcznh5p5nklpjm644epuh3pzc ==> Installing zlib-1.2.12-c4go4gxlcznh5p5nklpjm644epuh3pzc ... Fetch: 0.00s. Build: 0.88s. Total: 0.88s. [+] /tmp/tmp.b1eTyAOe85/store/linux-ubuntu20.04-zen2/gcc-10.3.0/zlib-1.2.12-c4go4gxlcznh5p5nklpjm644epuh3pzc spack -e . install --only-concrete --only=package /sbzszb7v557ohyd6c2ekirx2t3ctxfxp && touch sbzszb7v557ohyd6c2ekirx2t3ctxfxp ==> Installing pkgconf-1.8.0-sbzszb7v557ohyd6c2ekirx2t3ctxfxp ... Fetch: 0.00s. Build: 3.96s. Total: 3.96s. [+] /tmp/tmp.b1eTyAOe85/store/linux-ubuntu20.04-zen2/gcc-10.3.0/pkgconf-1.8.0-sbzszb7v557ohyd6c2ekirx2t3ctxfxp ``` For Perl, at least for me, using `make -j16` versus `spack -e . install -j16` speeds up the builds from 3m32.623s to 2m22.775s, as some configure scripts run in parallel. Another nice feature is you can do Makefile "metaprogramming" and depend on packages built by Spack. This example fetches all sources (in parallel) first, print a message, and only then build packages (in parallel). ```Makefile SPACK ?= spack .PHONY: env all: env spack.lock: spack.yaml $(SPACK) -e . concretize -f env.mk: spack.lock $(SPACK) -e . env depfile -o $@ --make-target-prefix spack fetch: spack/fetch @echo Fetched all packages && touch $@ env: fetch spack/env @echo This executes after the environment has been installed clean: rm -rf spack/ env.mk spack.lock ifeq (,$(filter clean,$(MAKECMDGOALS))) include env.mk endif ```
2022-05-05package audit: ensure stand-alone test method not include in build-phase ↵Tamara Dahlgren2-0/+20
testing (#30352)
2022-05-05spack external find: add search path customization (#30479)Greg Becker1-2/+7
2022-05-04bugfix: fix filter_compiler_wrappers for Cray compiler (#29786)Sergey Kosukhin1-3/+22
2022-05-04Remove deprecated "--run-tests" option of "spack install" (#30461)Massimiliano Culpo2-9/+1
2022-05-03Simplify fixture used to test `spack info` (#30456)Massimiliano Culpo1-18/+10
2022-05-03OpenCV and OpenBLAS: add external find support (#30240)Brian Van Essen3-8/+20
Added support for finding the OpenCV package via the find external command. Included support for identifying variants based on available shared libraries. Added support to finding the OpenBLAS package via the find external command. Enabled packages to show that they can be discovered via the find external command in the info message. Updated the OpenCV and OpenBLAS packages to use the extensible search mechanism for library extensions on multiple OS platforms. Corrected how find externals works on Darwin for OpenCV and OpenBLAS to accommodate that the version numbers are placed before the file extension instead of after it, as on Linux. Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-05-02spack ci: Fix typo in error message (#30438)Ken Raffenetti1-1/+1
2022-05-01bugfix: don't calculate `spack tutorial` version from version infoTodd Gamblin1-1/+1
A recent switch to the way we do `develop` versioning broke this. We should hard-code the latest tutorial version.
2022-04-29spack.yaml: add concretizer.yaml to the schema (#30376)Massimiliano Culpo1-0/+2
2022-04-28Add command for reading JSON-based DB description (now with more tests) (#29652)Peter Scheibel14-53/+764
This is an amended version of https://github.com/spack/spack/pull/24894 (reverted in https://github.com/spack/spack/pull/29603). https://github.com/spack/spack/pull/24894 broke all instances of `spack external find` (namely when it is invoked without arguments/options) because it was mandating the presence of a file which most systems would not have. This allows `spack external find` to proceed if that file is not present and adds tests for this. - [x] Add a test which confirms that `spack external find` successfully reads a manifest file if present in the default manifest path --- Original commit message --- Adds `spack external read-cray-manifest`, which reads a json file that describes a set of package DAGs. The parsed results are stored directly in the database. A user can see these installed specs with `spack find` (like any installed spec). The easiest way to use them right now as dependencies is to run `spack spec ... ^/hash-of-external-package`. Changes include: * `spack external read-cray-manifest --file <path/to/file>` will add all specs described in the file to Spack's installation DB and will also install described compilers to the compilers configuration (the expected format of the file is described in this PR as well including examples of the file) * Database records now may include an "origin" (the command added in this PR registers the origin as "external-db"). In the future, it is assumed users may want to be able to treat installs registered with this command differently (e.g. they may want to uninstall all specs added with this command) * Hash properties are now always preserved when copying specs if the source spec is concrete * I don't think the hashes of installed-and-concrete specs should change and this was the easiest way to handle that * also specs that are concrete preserve their `.normal` property when copied (external specs may mention compilers that are not registered, and without this change they would fail in `normalize` when calling `validate_or_raise`) * it might be this should only be the case if the spec was installed - [x] Improve testing - [x] Specifically mark DB records added with this command (so that users can do something like "uninstall all packages added with `spack read-external-db`) * This is now possible with `spack uninstall --all --origin=external-db` (this will remove all specs added from manifest files) - [x] Strip variants that are listed in json entries but don't actually exist for the package
2022-04-28ASP-based solver: discard unknown packages from reuse (#30357)Massimiliano Culpo2-3/+25
* ASP-based solver: discard unknown packages from reuse This is an add-on to #28259 that cover for the case of a single package.py being removed from a repository, rather than an entire custom repository being removed. * Add unit test
generated by cgit v1.2.3-60-g2f50 (git 2.42.0) at 2024-07-04 20:41:44 +0000