summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2024-11-01build(deps): bump docutils from 0.20.1 to 0.21.2 in /lib/spack/docs (#45592)dependabot[bot]1-1/+1
Bumps [docutils](https://docutils.sourceforge.io) from 0.20.1 to 0.21.2. --- updated-dependencies: - dependency-name: docutils dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-31database.py: remove process unsafe update_explicit (#47358)Harmen Stoppels3-24/+7
Fixes an issue reported where `spack env depfile` + `make -j` would non-deterministically refuse to mark all environment roots explicit. `update_explicit` had the pattern ```python rec = self._data[key] with self.write_transaction(): rec.explicit = explicit ``` but `write_transaction` may reinitialize `self._data`, meaning that mutating `rec` won't mutate `self._data`, and the changes won't be persisted. Instead, use `mark` which has a correct implementation. Also avoids the essentially incorrect early return in `update_explicit` which is a pattern I don't think belongs in database.py: it branches on possibly stale data to realize there is nothing to change, but in reality it requires a write transaction to know that for a fact, but that would defeat the purpose. So, leave this optimization to the call site.
2024-10-31Concretize reuse: reuse specs from environment (#45139)kwryankrattiger7-13/+279
The already concrete specs in an environment are now among the reusable specs for the concretizer. This includes concrete specs from all include_concrete environments. In addition to this change to the default reuse, `environment` is added as a reuse type for the concretizer config. This allows users to specify: spack: concretizer: # Reuse from this environment (including included concrete) but not elsewhere reuse: from: - type: environment # or reuse from only my_env included environment reuse: from: - type: environment: my_env # or reuse from everywhere reuse: true If reuse is specified from a specific environment, only specs from that environment will be reused. If the reused environment is not specified via include_concrete, the concrete specs will be retried at concretization time to be reused. Signed-off-by: Ryan Krattiger <ryan.krattiger@kitware.com> Co-authored-by: Gregory Becker <becker33@llnl.gov>
2024-10-31Docs: remove reference to pyspack (#47346)Adam J. Stewart1-4/+0
2024-10-30hooks: run in clear, fixed order (#47329)Harmen Stoppels2-32/+25
Currently the order in which hooks are run is arbitrary. This can be fixed by sorted(list_modules(...)) but I think it is much more clear to just have a static list. Hooks are not extensible other than modifying Spack code, which means it's unlikely people maintain custom hooks since they'd have to fork Spack. And if they fork Spack, they might as well add an entry to the list when they're continuously rebasing.
2024-10-30types: remove singleton union in globals (#47282)Harmen Stoppels4-29/+14
2024-10-30add std_pip_args global to the audit list (#47320)Harmen Stoppels1-1/+1
2024-10-30Spec.__str__: use full hash (#47322)Harmen Stoppels1-1/+1
The idea is that `spack -e env add ./concrete-spec.json` would list the full hash in the specs, so that (a) it's not ambiguous and (b) it could in principle results in constant time lookup instead of linear time substring match in large build caches.
2024-10-30Spec.dependencies: allow to filter on virtuals (#47284)Massimiliano Culpo3-49/+96
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2024-10-30Fix malformed RST link in documentation (#47309)Alex Hedges1-1/+1
2024-10-29llnl.util.lang: add classes to help with deprecations (#47279)Massimiliano Culpo2-0/+92
* Add a descriptor to have a class level constant This descriptor helps intercept places where we set a value on instances. It does not really behave like "const" in C-like languages, but is the simplest implementation that might still be useful. * Add a descriptor to deprecate properties/attributes of an object This descriptor is used as a base class. Derived classes may implement a factory to return an adaptor to the attribute being deprecated. The descriptor can either warn, or raise an error, when usage of the deprecated attribute is intercepted. --------- Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2024-10-29bootstrap: do not consider source when metadata file missing (#47278)Harmen Stoppels1-3/+6
2024-10-29std_meson_args: deprecate (#47259)Harmen Stoppels1-1/+1
2024-10-29builder.py: builder_cls should be associated to spack.pkg module (#47269)Harmen Stoppels2-5/+14
2024-10-29directives_meta.py: use startswith to test module part of spack.pkg (#47270)Harmen Stoppels1-1/+2
2024-10-28cmd/solve: use interface from cmd/spec (#47182)Greg Becker2-59/+20
Currently, `spack solve` has different spec selection semantics than `spack spec`. `spack solve` currently does not allow specifying a single spec when an environment is active. This PR modifies `spack solve` to inherit the interface from `spack spec`, and to use the same spec selection logic. This will allow for better use of `spack solve --show opt` for debugging. --------- Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2024-10-27Remove a few redundant imports (#47250)Harmen Stoppels6-15/+8
* remove self-imports * remove unused imports
2024-10-27explicit splice: do not fail for bad config replacement if target not ↵Greg Becker2-2/+42
matched (#46925) Originally, concretization failed if the splice config points to an invalid replacement. This PR defers the check until we know the splice is needed, so that irrelevant splices with bad config cannot stop concretization. While I was at it, I improved an error message from an assert to a ValueError.
2024-10-26Add `-t` short option for `spack --backtrace` (#47227)Todd Gamblin1-0/+1
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
2024-10-25Normalize Spack Win entry points (#38648)John W. Parent1-2/+0
* Normalize Spack Win entrypoints Currently Spack has multiple entrypoints on Windows that in addition to differing from *nix implementations, differ from shell to shell on Windows. This is a bit confusing for new users and in general unnecessary. This PR adds a normal setup script for the batch shell while preserving the previous "click from file explorer for spack shell" behavior. Additionally adds a shell title to both powershell and cmd letting users know this is a Spack shell * remove doskeys
2024-10-25Intel/Oneapi compilers: suppress warnings when using Cray wrappers (#47046)Dom Heinzeller2-6/+8
#44588 we added logic to suppress deprecation warnings for the Intel classic compilers. This depended on matching against * The compiler names (looking for icc, icpc, ifort) * The compiler version When using an Intel compiler with fortran wrappers, the first check always fails. To support using the fortran wrappers (in combination with the classic Intel compilers), we remove the first check and suppress if just the version matches. This works because: * The newer compilers like icx can handle (ignore) the flags that suppress deprecation warnings * The Cray wrappers pass the underlying compiler version (e.g. they report what icc would report)
2024-10-25ensure write_fd.close() isn't called when sys.std* cannot be redirectedHarmen Stoppels1-6/+7
2024-10-25Avoid a socket to communicate effectively a bitHarmen Stoppels1-3/+3
2024-10-25Replace MultiProcessFd with Connection objectsHarmen Stoppels2-134/+68
Connection objects are Python version, platform and multiprocessing start method independent, so better to use those than a mix of plain file descriptors and inadequate guesses in the child process whether it was forked or not. This also allows us to delete the now redundant MultiProcessFd class, hopefully making things a bit easier to follow.
2024-10-25file_cache.py: allow read transaction on uninitialized cache (#47212)Harmen Stoppels2-40/+58
This allows the following ```python cache.init_entry("my/cache") with cache.read_transaction("my/cache") as f: data = f.read() if f is not None else None ``` mirroring `write_transaction`, which returns a tuple `(old, new)` where `old` is `None` if the cache file did not exist yet. The alternative that requires less defensive programming on the call site would be to create the "old" file upon first read, but I did not want to think about how to safely atomically create the file, and it's not unthinkable that an empty file is an invalid format (for instance the call site may expect a json file, which requires at least {} bytes).
2024-10-25Add spack short version in config variables (#47016)Jordan Galby4-14/+35
2024-10-24Env help: expand and refine subcommand help and descriptions (#47089)Tamara Dahlgren1-62/+97
This PR is in response to a question in the `environments` slack channel (https://spackpm.slack.com/archives/CMHK7MF51/p1729200068557219) about inadequate CLI help/documentation for one specific subcommand. This PR uses the approach I took for the descriptions and help for `spack test` subcommands. Namely, I use the first line of the relevant docstring as the description, which is shown per subcommand in `spack env -h`, and the entire docstring as the help. I then added, where it seemed appropriate, help. I also tweaked argument docstrings to tighten them up, make consistent with similar arguments elsewhere in the command, and elaborate when it seemed important. (The only subcommand I didn't touch is `loads`.) For example, before: ``` $ spack env update -h usage: spack env update [-hy] env positional arguments: env name or directory of the environment to activate optional arguments: -h, --help show this help message and exit -y, --yes-to-all assume "yes" is the answer to every confirmation request ``` After the changes in this PR: ``` $ spack env update -h usage: spack env update [-hy] env update the environment manifest to the latest schema format update the environment to the latest schema format, which may not be readable by older versions of spack a backup copy of the manifest is retained in case there is a need to revert this operation positional arguments: env name or directory of the environment optional arguments: -h, --help show this help message and exit -y, --yes-to-all assume "yes" is the answer to every confirmation request ``` --------- Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2024-10-24compilers.yaml: require list of strings for modules (#47197)Harmen Stoppels2-2/+5
2024-10-24Restore bold uncolored font face (#47108)Kyle Knoepfel1-1/+3
Commit aa0825d642cfa285f5f62761a0e23dc1e511d056 accidentally added a semicolon to the ANSI escape sequence even if the color code was `None` or unknown, breaking the bold, uncolored font-face. This PR restores the old behavior. --------- Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2024-10-24bootstrap: add clingo 3.13 binaries and more (#47126)Harmen Stoppels3-8/+4
2024-10-24Improve `Database.query*` methods (#47116)Massimiliano Culpo8-128/+181
* Add type hints to all query* methods * Inline docstrings * Change defaults from `any` to `None` so they can be type hinted in old Python * Pre-filter on given hashes instead of iterating over all db specs * Fix a bug where the `--origin` option of uninstall had no effect * Fix a bug where query args were not applied when searching by concrete spec Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2024-10-23env: continue to mark non-roots as implicitly installed on partial env ↵Harmen Stoppels2-11/+24
installs (#47183) Fixes a change in behavior/bug in 70412612c79af495fb2b2223edac4bd5a70a813a, where partial environment installs would mark the selected spec as explicitly installed, even if it was not a root of the environment. The desired behavior is that roots by definition are the to be explicitly installed specs. The specs on the `spack -e ... install x` command line are just filters for partial installs, so leave them implicitly installed if they aren't roots.
2024-10-23ci: Remove deprecated logic from the ci module (#47062)Scott Wittenburg7-1059/+171
ci: Remove deprecated logic from the ci module Remove the following from the ci module, schema, and tests: - deprecated ci stack and handling of old ci config - deprecated mirror handling logic - support for artifacts buildcache - support for temporary storage url
2024-10-23mypy: work around typing issues with `functools.partial` (#47160)Todd Gamblin1-5/+9
2024-10-23build(deps): bump sphinx from 7.4.7 to 8.1.3 in /lib/spack/docs (#47159)dependabot[bot]1-1/+1
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 7.4.7 to 8.1.3. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/v8.1.3/CHANGES.rst) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v7.4.7...v8.1.3) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-23Remove spurious warning, introduced in #46992 (#47152)Massimiliano Culpo1-1/+1
fixes #47135 Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2024-10-22build(deps): bump black from 24.8.0 to 24.10.0 in /lib/spack/docs (#47118)dependabot[bot]1-1/+1
Bumps [black](https://github.com/psf/black) from 24.8.0 to 24.10.0. - [Release notes](https://github.com/psf/black/releases) - [Changelog](https://github.com/psf/black/blob/main/CHANGES.md) - [Commits](https://github.com/psf/black/compare/24.8.0...24.10.0) --- updated-dependencies: - dependency-name: black dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-22spack arch: add --family --generic flags (#47078)Harmen Stoppels1-1/+17
This allows users to do: ``` spack install ... target=$(spack arch --target --family) spack install ... arch=$(spack arch --family) spack install ... target=$(spack arch --target --generic) spack install ... arch=$(spack arch --generic) ``` Deprecate `--generic-target` in favor of `--generic --target`
2024-10-22buildcache: recognize . and .. as paths instead of names (#47105)Harmen Stoppels3-11/+19
2024-10-21avoid double closing of fd in sub-processes (#47035)Peter Scheibel2-23/+22
Both `multiprocessing.connection.Connection.__del__` and `io.IOBase.__del__` called `os.close` on the same file descriptor. As of Python 3.13, this is an explicit warning. Ensure we close once by usef `os.fdopen(..., closefd=False)`
2024-10-21Devtools darwin (#46910)Tom Scogland1-1/+5
* stacks: add a stack for devtools on darwin After getting this whole mess building on darwin, let's keep it that way, and maybe make it so we have some non-ML darwin binaries in spack as well. * reuse: false for devtools * dtc: fix darwin dylib name and id On mac the convention is `lib<name>.<num>.dylib`, while the makefile creates a num suffixed one by default. The id in the file is also a local name rather than rewritten to the full path, this fixes both problems. * node-js: make whereis more deterministic * relocation(darwin): catch Mach-O load failure The MachO library can throw an exception rather than return no headers, this happened in an elf file in the test data of go-bootstrap. Trying catching the exception and moving on for now. May also need to look into why we're trying to rewrite an elf file. * qemu: add darwin flags to clear out warnings There's a build failure for qemu in CI, but it's invisible because of the immense mass of warning output. Explicitly specify the target macos version and remove the extraneous unknown-warning-option flag. * dtc: libyaml is also a link dependency libyaml is required at runtime to run the dtc binary, lack of it caused the ci for qemu to fail when the library wasn't found.
2024-10-21bootstrap: handle a new edge case of binary python packages with missing ↵Harmen Stoppels4-14/+37
python-venv (#47094) relevant for clingo installed without gcc-runtime and python-venv, which is done for good reasons.
2024-10-21Fix broken `spack find -u` (#47102)Massimiliano Culpo5-17/+37
fixes #47101 The bug was introduced in #33495, where `spack find was not updated, and wasn't caught by unit tests. Now a Database can accept a custom predicate to select the installation records. A unit test is added to prevent regressions. The weird convention of having `any` as a default value has been replaced by the more commonly used `None`. Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2024-10-21Modules suffixes config are now spec format strings (#38411)Jordan Galby4-4/+22
2024-10-21Docs: clarify include path options (#47083)Tamara Dahlgren1-0/+3
2024-10-19spack external find: show backtrace on error when --backtrace (#47082)Harmen Stoppels3-11/+16
2024-10-19build(deps): bump python-levenshtein in /lib/spack/docs (#46494)dependabot[bot]1-1/+1
Bumps [python-levenshtein](https://github.com/rapidfuzz/python-Levenshtein) from 0.25.1 to 0.26.0. - [Release notes](https://github.com/rapidfuzz/python-Levenshtein/releases) - [Changelog](https://github.com/rapidfuzz/python-Levenshtein/blob/main/HISTORY.md) - [Commits](https://github.com/rapidfuzz/python-Levenshtein/compare/v0.25.1...v0.26.0) --- updated-dependencies: - dependency-name: python-levenshtein dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-19rtd: bump sphinx-rtd-theme to 3.0.1 (#47002)Wouter Deconinck1-1/+1
2024-10-18Build logic fix: reorder definition of package module variables (#46992)John W. Parent2-4/+41
#44327 made sure to always run `set_package_py_globals` on all packages before running `setup_dependent_package` for any package, so that packages implementing the latter could depend on variables like `spack_cc` being defined. This ran into an undocumented dependency: `std_cmake_args` is set in `set_package_py_globals` and makes use of `cmake_prefix_paths` (if it is defined in the package); `py-torch`es implementation of `cmake_prefix_paths` depends on a variable set by `setup_dependent_package` (`python_platlib`). This generally restores #44327, and corrects the resulting issue by moving assignment of `std_cmake_args` to after both actions have been run.
2024-10-18implement prefix property for OneAPI compiler (#47066)Sean Koyama1-0/+20