summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2023-07-24Always set workflow:rules in spack ci pipelines (#38921)Jonathon Anderson2-13/+20
2023-07-24Fix typo: binary_caches.rst (#39064)Ashwin Kumar Karnad1-1/+1
2023-07-24Don't add `.spack/binary_distribution` twice to the tarball when ↵Harmen Stoppels2-2/+44
re-distributing (#39042) Previously, spack would list the ./spack/binary_distribution file twice when pushing a package that was installed from a binary tarball itself.
2023-07-24Fix broken `sanitize_file_path` (#38926)Harmen Stoppels6-35/+52
The sanitization function is completely bogus as it tries to replace / on unix after ... splitting on it. The way it's implemented is very questionable: the input is a file name, not a path. It doesn't make sense to interpret the input as a path and then make the components valid -- you'll interpret / in a filename as a dir separator. It also fails to deal with path components that contain just unsupported characters (resulting in empty component). The correct way to deal with this is to have a function that takes a potential file name and replaces unsupported characters. I'm not going to fix the other issues on Windows, such as reserved file names, but left a note, and hope that @johnwparent can fix that separately. (Obviously we wouldn't have this problem at all if we just fixed the filename in a safe way instead of trying to derive something from the url; we could use the content digest when available for example)
2023-07-22Implement fish completion (#29549)百地 希留耶5-165/+563
* commands: provide more information to Command * fish: Add script to generate fish completion * fish: auto prepend `spack` command to avoid duplication * fish: impove completion generation code readability * commands: replace match-case with if-else * fish: fix optspec variable name prefix * fish: fix return value in get_optspecs * fish: fix return value in get_optspecs * format: split long line and trim trailing space * bugfix: replace f-string with interpolation * fish: compete more specs and some fixes * fish: complete hash spec starts with / * fish: improve compatibility * style: trim trailing whitespace * commands: add fish to update args and update tests * commands: add fish completion file * style: merge imports * fish: source completion in setup-env * fish: caret only completes dependencies * fish: make sure we always get same order of output * fish: spack activate only show installed packages that have extensions * fish: update completion file * fish: make dict keys sorted * Blacken code * Fix bad merge * Undo style changes to setup-env.fish * Fix unit tests * Style fix * Compatible with fish_indent * Use list for stability of order * Sort one more place * Sort more things * Sorting unneeded * Unsort * Print difference * Style fix * Help messages need quotes * Arguments to -a must be quoted * Update types * Update types * Update types * Add type hints * Change order of positionals * Always expand help * Remove shared base class * Fix type hints * Remove platform-specific choices * First line of help only * Remove unused maps * Remove suppress * Remove debugging comments * Better quoting * Fish completions have no double dash * Remove test for deleted class * Fix grammar in header file * Use single quotes in most places * Better support for remainder nargs * No magic strings * * and + can also complete multiple * lower case, no period --------- Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2023-07-21Remove spack.repo.IndexError (#39029)Massimiliano Culpo1-4/+0
This exception is never used and overrides a built-in.
2023-07-21binary_caches.rst: fix typo (#39030)Ashwin Kumar Karnad1-1/+1
2023-07-20nag: update the versioning scheme (#35457)Sergey Kosukhin2-2/+11
* nag: append build number to the version * nag: add version 7.1.7125 * nag: deprecate unavailable versions
2023-07-20spack.util.lock: add type-hints, remove **kwargs in method signatures (#39011)Massimiliano Culpo4-23/+37
2023-07-19Respect custom user store when bootstrapping (#39001)Massimiliano Culpo3-1/+26
The user store is lazily evaluated. The change in #38975 made it such that the first evaluation was happening in the middle of swapping to user configuration. Ensure we construct the user store before that.
2023-07-19modules: use curly braces to enclose value in Tcl modulefile (#38375)Xavier Delaruelle3-33/+65
Use curly braces instead of quotes to enclose value or text in Tcl modulefile. Within curly braces Tcl special characters like [, ] or $ are treated verbatim whereas they are evaluated within quotes. Curly braces is Tcl recommended way to enclose verbatim content [1]. Note: if curly braces charaters are used within content, they must be balanced. This point has been checked against current repository and no unbalanced curly braces has been spotted. Fixes #24243 [1] https://wiki.tcl-lang.org/page/Tcl+Minimal+Escaping+Style
2023-07-19Fetch & patch: actually acquire stage lock, and many more issues (#38903)Harmen Stoppels17-219/+210
* Fetching patches wouldn't result in acquiring a stage lock during install * The installer would acquire a stage lock *after* fetching instead of before, leading to races * The name of the stage for patches was random, so on build failure (where stage dirs are not removed), these directories would continue to exist after a second successful install. * There was this redundant "composite fetch" object -- there's already a composite stage. Remove this. * For some reason we do *double* shasum validation of patches, before and after compression -- that's just too much? I removed it.
2023-07-19drop redundant rpaths post install (#38976)Harmen Stoppels2-0/+152
Spack heuristically adds `<install prefix>/lib` and `<install prefix>/lib64` as rpath entries, as it doesn't know what the install dir is going to be ahead of the build. This PR cleans up non-existing, absolute paths[^1], which 1. avoids redundant stat calls at runtime 2. drops redundant rpaths in `patchelf`, making it relocatable -- you don't need patchelf recursively then. [^1]: It also removes relative paths not starting with `$` (so, `$ORIGIN/../lib` is retained -- we _could_ interpolate `$ORIGIN`, but that's hard to get right when symlinks have to be taken into account). Relative paths _are_ supported in glibc, but are relative to _the current working directory_, which is madness, and it would be better to drop those paths.
2023-07-19llnl.util.lock: add type-hints (#38977)Massimiliano Culpo3-71/+114
Also uppercase global variables in the module
2023-07-19Remove `LazyReference` from code (#38944)Massimiliano Culpo72-425/+376
A LazyReference object is a reference to an attribute of a lazily evaluated singleton. Its only purpose is to let developers use shorter names to refer to such attribute. This class does more harm than good, as it obfuscates the fact that we are using the attribute of a global object. Also, it can easily go out of sync with the singleton it refers to if, for instance, the singleton is updated but the references are not. This commit removes the LazyReference class entirely, and access the attributes explicitly passing through the global value to which they are attached.
2023-07-18Windows testing: enable tests for installer components (#36970)markus-ferrell1-8/+0
These tests now work without any changes to core. Furthermore, it is surprising that they had to be disabled (at least, as long as the installer.py tests are run on Windows: these tests are more-basic and their functionality would have been exercised automatically).
2023-07-18Ensure the bootstrap store has a padding length of zero (#38975)Massimiliano Culpo2-17/+26
Without this PR, padded length was propagating from user configuration to bootstrap configuration, and was causing the issue reported in #38963
2023-07-18Windows testing: enable bootstrap test (#36972)markus-ferrell1-2/+0
2023-07-18test_clear_failures_success: run on Windows too (#36792)markus-ferrell1-2/+3
2023-07-18binary cache docs: remove redundant flag and comment (#38960)Harmen Stoppels1-5/+1
2023-07-18Windows testing: enable architecture test (#36973)markus-ferrell1-2/+0
Works out of the box: remove skip.
2023-07-18Windows testing: enable "spack clean" tests (#36840)markus-ferrell1-3/+0
They work out out of the box on windows. Simply removing skips.
2023-07-18buildcache push: make --allow-root the default and deprecate the option (#38878)Harmen Stoppels14-218/+36
Without --allow-root spack cannot push binaries that contain paths in binaries. This flag is almost always needed, so there is no point of requiring users to spell it out. Even without --allow-root, rpaths would still have to be patched, so the flag is not there to guarantee binaries are not modified on install. This commit makes --allow-root the default, and drops the code required for it. It also deprecates `spack buildcache preview`, since the command made sense only with --allow-root. As a side effect, Spack no longer depends on binutils for relocation
2023-07-18Pin Spack dev dependencies on RtD (#38950)Massimiliano Culpo1-0/+5
2023-07-18Fix default construction of locks (#38953)Massimiliano Culpo1-1/+1
This fixes a typo introduced in a refactor
2023-07-18modules: add support for conflict in lua modulefile (#36701)Xavier Delaruelle8-59/+116
Add support for conflict directives in Lua modulefile like done for Tcl modulefile. Note that conflicts are correctly honored on Lmod and Environment Modules <4.2 only if mutually expressed on both modulefiles that conflict with each other. Migrate conflict code from Tcl-specific classes to the common part. Add tests for Lmod and split the conflict test case in two. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-07-18"spack develop": always pull full history for git repos (#38343)Peter Scheibel4-9/+66
2023-07-17Decompression: fix naming issues (#37749)Dan Lipsa2-35/+76
* When using system tools to unpack a .gz file, the input file needs a different name than the output file. Normally, we generate this new name by stripping off the .gz extension off of the file name. This was not sufficient if the file name did not have an extension, so we temporarily rename the file in that case. * When using system tar utility to untar on Windows, we were (erroneously) skipping the actual untar step if the filename was lacking a .tar extension * For foo.txz, we were not changing the extension of the decompressed file (i.e. we would decompress foo.txz to foo.txz). This did not cause any problems, but is confusing, so has been updated such that the output filename reflects its decompressed state (i.e. foo.tar). * Added test for strip_compression_extension * Update test_native_unpacking to test each archive type with and without an extension as part of the file name (i.e. we test "foo.tar.gz", but also make sure we decompress properly if it is named "foo").
2023-07-17Fix wrong StageComposite keep override (#38938)Harmen Stoppels2-2/+38
`Stage(keep=True)` was ignored when put in a composite that doesn't override the value.
2023-07-17Fix python import tests (#38928)Manuela Kuhn1-1/+1
Running `spack test run <python package>` resulted in the error ``` 'str' object is not callable ``` because the python executable was not set correctly.
2023-07-17Lock, database and store don't need global configuration on construction ↵Massimiliano Culpo24-337/+431
(#33495) Lock objects can now be instantiated independently, without being tied to the global configuration. The same is true for database and store objects. The database __init__ method has been simplified to take a single lock configuration object. Some common lock configurations (e.g. NO_LOCK or NO_TIMEOUT) have been named and are provided as globals. The use_store context manager keeps the configuration consistent by pushing and popping an internal scope. It can also be tuned by passing extra data to set up e.g. upstreams or anything else that might be related to the store.
2023-07-17Update new PythonPackage template to prefer --config-settings (#38918)Adam J. Stewart3-17/+12
2023-07-17Environment/depfile: fix bug with Git hash versions (attempt #2) (#37560)Peter Scheibel2-12/+91
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-07-17Minimal cleanup of a few tests in `test/packaging.py` (#38880)Massimiliano Culpo1-162/+125
* Minimal cleanup of a few tests in packaging.py * Use f-strings
2023-07-14Remove unused context manager (#38897)Harmen Stoppels1-8/+0
2023-07-14Fix issue on cray with super call (#38895)Rocco Meli1-1/+1
2023-07-13Bugfix/spack spec: read and use the environment concretizer:unification ↵Tamara Dahlgren2-4/+62
option (#38248) * Bugfix: spack.yaml concretizer:unify needs to be read and used * Optional: add environment test to ensure configuration scheme is used * Activate environment in unit tests A more proper solution would be to keep an environment instance configuration as an attribute, but that is a bigger refactor * Delay evaluation of Environment.unify * Slightly simplify unit tests --------- Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-07-13spack buildcache push: improve argparse (#38876)Harmen Stoppels1-8/+11
2023-07-13mirrors: distinguish between source/binary mirror; simplify schema (#34523)Harmen Stoppels11-298/+469
Allow the following formats: ```yaml mirrors: name: <url> ``` ```yaml mirrors: name: url: s3://xyz access_pair: [x, y] ``` ```yaml mirrors: name: fetch: http://xyz push: url: s3://xyz access_pair: [x, y] ``` And reserve two new properties to indicate the mirror type (e.g. mirror.spack.io is a source mirror, not a binary cache) ```yaml mirrors: spack-public: source: true binary: false url: https://mirror.spack.io ```
2023-07-13spack audit: allow skipping version checks from package.py (#28372)Massimiliano Culpo1-1/+12
A few packages have version directives evaluated within if statements, conditional on the value of `platform.platform()`. Sometimes there are no cases for e.g. platform=darwin and that causes a lot of spurious failures with version existence audits. This PR allows expressing conditions to skip version existence checks in audits and avoid these spurious reports.
2023-07-13Add type hints to spack.installer (#38872)Harmen Stoppels4-623/+670
2023-07-13Add missing space in error msg (#38863)Adam J. Stewart1-1/+1
2023-07-13Standardize subcommand help strings (#38804)Adam J. Stewart42-348/+295
### Rationale While working on #29549, I noticed a lot of inconsistencies in our argparse help messages. This is important for fish where these help messages end up as descriptions in the tab completion menu. See https://github.com/spack/spack/pull/29549#issuecomment-1627596477 for some examples of longer or more stylized help messages. ### Implementation This PR makes the following changes: - [x] help messages start with a lowercase letter. - [x] Help messages do not end with a period - [x] the first line of a help message is short and simple longer text is separated by an empty line - [x] "help messages do not use triple quotes" """(except docstrings)""" - [x] Parentheses not needed for string concatenation inside function call - [x] Remove "..." "..." string concatenation leftover from black reformatting - [x] Remove Sphinx argument docs from help messages The first 2 choices aren't very controversial, and are designed to match the syntax of the `--help` flag automatically added by argparse. The 3rd choice is more up for debate, and is designed to match our package/module docstrings. The 4th choice is designed to avoid excessive newline characters and indentation. We may actually want to go even further and disallow docstrings altogether. ### Alternatives Choice 3 in particular has a lot of alternatives. My goal is solely to ensure that fish tab completion looks reasonable. Alternatives include: 1. Get rid of long help messages, only allow short simple messages 2. Move longer help messages to epilog 3. Separate by 2 newline characters instead of 1 4. Separate by period instead of newline. First sentence goes into tab completion description The number of commands with long help text is actually rather small, and is mostly relegated to `spack ci` and `spack buildcache`. So 1 isn't actually as ridiculous as it sounds. Let me know if there are any other standardizations or alternatives you would like to suggest.
2023-07-12containers: retain shallow git data (#37734)Jonathon Anderson2-7/+13
2023-07-12Add path to MPI executables to ^mpi dependents (#35758)Michael Fink1-3/+7
2023-07-12vtk-m: correct cuda_arch variant behavior (#38697)Vicente Bolea2-0/+48
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
2023-07-12installer: Improve status reporting (#37903)Michael Kuhn4-48/+64
Refactor `TermTitle` into `InstallStatus` and use it to show progress information both in the terminal title as well as inline. This also turns on the terminal title status by default. The inline output will look like the following after this change: ``` ==> Installing m4-1.4.19-w2fxrpuz64zdq63woprqfxxzc3tzu7p3 [4/4] ```
2023-07-10find: add `--hashes` shortcut for piping to other commands (#38663)Todd Gamblin1-0/+8
People frequently ask us how to pipe `spack find` output to other commands, and we tell them to do things like this: ```console $ spack find --format "/{hash}" | spack uninstall -ay ``` Sometimes users don't know about hash references and come up with potentially ambiguous formulations like this: ```console spack find --format {name}@{version}%{compiler} | spack uninstall -ay ``` Since this is a common enough thing to want to do, and to make it more obvious how, this PR adds a `-H` / `--hashes` as a shortcut, so you can now just do: ```console spack find -H | spack uninstall -ay ```
2023-07-09ci: Fix broken SPACK_CHECKOUT_VERSION (#38778)Jonathon Anderson2-17/+13
2023-07-09Don’t call spec.format in Database._get_matching_spec_key (#38792)Harmen Stoppels1-1/+15
`"%s" % spec` formats the spec with deps included, which produces sometimes KBs of data and is slow to run in pure Python. It can delay otherwise very short-lived read/write locks on the database. Discovered in #38762 where profile output showed about 2 seconds is spent in `spec.format`, which is significant overhead when using multiprocessing to install from binary cache in parallel (installation often takes <5s for small packages). With this change, `spec.format` no longer shows up in profile output. (This line hasn't changed since Spack v0.9 ;p) * move format() call to custom NoSuchSpecError exception * add a comment saying why, so we can eventually change `Spec.__str__`