summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2023-07-19Remove `LazyReference` from code (#38944)Massimiliano Culpo72-425/+376
A LazyReference object is a reference to an attribute of a lazily evaluated singleton. Its only purpose is to let developers use shorter names to refer to such attribute. This class does more harm than good, as it obfuscates the fact that we are using the attribute of a global object. Also, it can easily go out of sync with the singleton it refers to if, for instance, the singleton is updated but the references are not. This commit removes the LazyReference class entirely, and access the attributes explicitly passing through the global value to which they are attached.
2023-07-18Windows testing: enable tests for installer components (#36970)markus-ferrell1-8/+0
These tests now work without any changes to core. Furthermore, it is surprising that they had to be disabled (at least, as long as the installer.py tests are run on Windows: these tests are more-basic and their functionality would have been exercised automatically).
2023-07-18Ensure the bootstrap store has a padding length of zero (#38975)Massimiliano Culpo2-17/+26
Without this PR, padded length was propagating from user configuration to bootstrap configuration, and was causing the issue reported in #38963
2023-07-18Windows testing: enable bootstrap test (#36972)markus-ferrell1-2/+0
2023-07-18test_clear_failures_success: run on Windows too (#36792)markus-ferrell1-2/+3
2023-07-18binary cache docs: remove redundant flag and comment (#38960)Harmen Stoppels1-5/+1
2023-07-18Windows testing: enable architecture test (#36973)markus-ferrell1-2/+0
Works out of the box: remove skip.
2023-07-18Windows testing: enable "spack clean" tests (#36840)markus-ferrell1-3/+0
They work out out of the box on windows. Simply removing skips.
2023-07-18buildcache push: make --allow-root the default and deprecate the option (#38878)Harmen Stoppels14-218/+36
Without --allow-root spack cannot push binaries that contain paths in binaries. This flag is almost always needed, so there is no point of requiring users to spell it out. Even without --allow-root, rpaths would still have to be patched, so the flag is not there to guarantee binaries are not modified on install. This commit makes --allow-root the default, and drops the code required for it. It also deprecates `spack buildcache preview`, since the command made sense only with --allow-root. As a side effect, Spack no longer depends on binutils for relocation
2023-07-18Pin Spack dev dependencies on RtD (#38950)Massimiliano Culpo1-0/+5
2023-07-18Fix default construction of locks (#38953)Massimiliano Culpo1-1/+1
This fixes a typo introduced in a refactor
2023-07-18modules: add support for conflict in lua modulefile (#36701)Xavier Delaruelle8-59/+116
Add support for conflict directives in Lua modulefile like done for Tcl modulefile. Note that conflicts are correctly honored on Lmod and Environment Modules <4.2 only if mutually expressed on both modulefiles that conflict with each other. Migrate conflict code from Tcl-specific classes to the common part. Add tests for Lmod and split the conflict test case in two. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-07-18"spack develop": always pull full history for git repos (#38343)Peter Scheibel4-9/+66
2023-07-17Decompression: fix naming issues (#37749)Dan Lipsa2-35/+76
* When using system tools to unpack a .gz file, the input file needs a different name than the output file. Normally, we generate this new name by stripping off the .gz extension off of the file name. This was not sufficient if the file name did not have an extension, so we temporarily rename the file in that case. * When using system tar utility to untar on Windows, we were (erroneously) skipping the actual untar step if the filename was lacking a .tar extension * For foo.txz, we were not changing the extension of the decompressed file (i.e. we would decompress foo.txz to foo.txz). This did not cause any problems, but is confusing, so has been updated such that the output filename reflects its decompressed state (i.e. foo.tar). * Added test for strip_compression_extension * Update test_native_unpacking to test each archive type with and without an extension as part of the file name (i.e. we test "foo.tar.gz", but also make sure we decompress properly if it is named "foo").
2023-07-17Fix wrong StageComposite keep override (#38938)Harmen Stoppels2-2/+38
`Stage(keep=True)` was ignored when put in a composite that doesn't override the value.
2023-07-17Fix python import tests (#38928)Manuela Kuhn1-1/+1
Running `spack test run <python package>` resulted in the error ``` 'str' object is not callable ``` because the python executable was not set correctly.
2023-07-17Lock, database and store don't need global configuration on construction ↵Massimiliano Culpo24-337/+431
(#33495) Lock objects can now be instantiated independently, without being tied to the global configuration. The same is true for database and store objects. The database __init__ method has been simplified to take a single lock configuration object. Some common lock configurations (e.g. NO_LOCK or NO_TIMEOUT) have been named and are provided as globals. The use_store context manager keeps the configuration consistent by pushing and popping an internal scope. It can also be tuned by passing extra data to set up e.g. upstreams or anything else that might be related to the store.
2023-07-17Update new PythonPackage template to prefer --config-settings (#38918)Adam J. Stewart3-17/+12
2023-07-17Environment/depfile: fix bug with Git hash versions (attempt #2) (#37560)Peter Scheibel2-12/+91
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-07-17Minimal cleanup of a few tests in `test/packaging.py` (#38880)Massimiliano Culpo1-162/+125
* Minimal cleanup of a few tests in packaging.py * Use f-strings
2023-07-14Remove unused context manager (#38897)Harmen Stoppels1-8/+0
2023-07-14Fix issue on cray with super call (#38895)Rocco Meli1-1/+1
2023-07-13Bugfix/spack spec: read and use the environment concretizer:unification ↵Tamara Dahlgren2-4/+62
option (#38248) * Bugfix: spack.yaml concretizer:unify needs to be read and used * Optional: add environment test to ensure configuration scheme is used * Activate environment in unit tests A more proper solution would be to keep an environment instance configuration as an attribute, but that is a bigger refactor * Delay evaluation of Environment.unify * Slightly simplify unit tests --------- Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-07-13spack buildcache push: improve argparse (#38876)Harmen Stoppels1-8/+11
2023-07-13mirrors: distinguish between source/binary mirror; simplify schema (#34523)Harmen Stoppels11-298/+469
Allow the following formats: ```yaml mirrors: name: <url> ``` ```yaml mirrors: name: url: s3://xyz access_pair: [x, y] ``` ```yaml mirrors: name: fetch: http://xyz push: url: s3://xyz access_pair: [x, y] ``` And reserve two new properties to indicate the mirror type (e.g. mirror.spack.io is a source mirror, not a binary cache) ```yaml mirrors: spack-public: source: true binary: false url: https://mirror.spack.io ```
2023-07-13spack audit: allow skipping version checks from package.py (#28372)Massimiliano Culpo1-1/+12
A few packages have version directives evaluated within if statements, conditional on the value of `platform.platform()`. Sometimes there are no cases for e.g. platform=darwin and that causes a lot of spurious failures with version existence audits. This PR allows expressing conditions to skip version existence checks in audits and avoid these spurious reports.
2023-07-13Add type hints to spack.installer (#38872)Harmen Stoppels4-623/+670
2023-07-13Add missing space in error msg (#38863)Adam J. Stewart1-1/+1
2023-07-13Standardize subcommand help strings (#38804)Adam J. Stewart42-348/+295
### Rationale While working on #29549, I noticed a lot of inconsistencies in our argparse help messages. This is important for fish where these help messages end up as descriptions in the tab completion menu. See https://github.com/spack/spack/pull/29549#issuecomment-1627596477 for some examples of longer or more stylized help messages. ### Implementation This PR makes the following changes: - [x] help messages start with a lowercase letter. - [x] Help messages do not end with a period - [x] the first line of a help message is short and simple longer text is separated by an empty line - [x] "help messages do not use triple quotes" """(except docstrings)""" - [x] Parentheses not needed for string concatenation inside function call - [x] Remove "..." "..." string concatenation leftover from black reformatting - [x] Remove Sphinx argument docs from help messages The first 2 choices aren't very controversial, and are designed to match the syntax of the `--help` flag automatically added by argparse. The 3rd choice is more up for debate, and is designed to match our package/module docstrings. The 4th choice is designed to avoid excessive newline characters and indentation. We may actually want to go even further and disallow docstrings altogether. ### Alternatives Choice 3 in particular has a lot of alternatives. My goal is solely to ensure that fish tab completion looks reasonable. Alternatives include: 1. Get rid of long help messages, only allow short simple messages 2. Move longer help messages to epilog 3. Separate by 2 newline characters instead of 1 4. Separate by period instead of newline. First sentence goes into tab completion description The number of commands with long help text is actually rather small, and is mostly relegated to `spack ci` and `spack buildcache`. So 1 isn't actually as ridiculous as it sounds. Let me know if there are any other standardizations or alternatives you would like to suggest.
2023-07-12containers: retain shallow git data (#37734)Jonathon Anderson2-7/+13
2023-07-12Add path to MPI executables to ^mpi dependents (#35758)Michael Fink1-3/+7
2023-07-12vtk-m: correct cuda_arch variant behavior (#38697)Vicente Bolea2-0/+48
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
2023-07-12installer: Improve status reporting (#37903)Michael Kuhn4-48/+64
Refactor `TermTitle` into `InstallStatus` and use it to show progress information both in the terminal title as well as inline. This also turns on the terminal title status by default. The inline output will look like the following after this change: ``` ==> Installing m4-1.4.19-w2fxrpuz64zdq63woprqfxxzc3tzu7p3 [4/4] ```
2023-07-10find: add `--hashes` shortcut for piping to other commands (#38663)Todd Gamblin1-0/+8
People frequently ask us how to pipe `spack find` output to other commands, and we tell them to do things like this: ```console $ spack find --format "/{hash}" | spack uninstall -ay ``` Sometimes users don't know about hash references and come up with potentially ambiguous formulations like this: ```console spack find --format {name}@{version}%{compiler} | spack uninstall -ay ``` Since this is a common enough thing to want to do, and to make it more obvious how, this PR adds a `-H` / `--hashes` as a shortcut, so you can now just do: ```console spack find -H | spack uninstall -ay ```
2023-07-09ci: Fix broken SPACK_CHECKOUT_VERSION (#38778)Jonathon Anderson2-17/+13
2023-07-09Don’t call spec.format in Database._get_matching_spec_key (#38792)Harmen Stoppels1-1/+15
`"%s" % spec` formats the spec with deps included, which produces sometimes KBs of data and is slow to run in pure Python. It can delay otherwise very short-lived read/write locks on the database. Discovered in #38762 where profile output showed about 2 seconds is spent in `spec.format`, which is significant overhead when using multiprocessing to install from binary cache in parallel (installation often takes <5s for small packages). With this change, `spec.format` no longer shows up in profile output. (This line hasn't changed since Spack v0.9 ;p) * move format() call to custom NoSuchSpecError exception * add a comment saying why, so we can eventually change `Spec.__str__`
2023-07-07Remove "node_compiler" from the list of unknown atoms (#38753)Massimiliano Culpo1-0/+1
2023-07-07Fix multiple quadratic complexity issues in environments (#38771)Harmen Stoppels3-63/+57
1. Fix O(n^2) iteration in `_get_overwrite_specs` 2. Early exit `get_by_hash` on full hash 3. Fix O(n^2) double lookup in `all_matching_specs` with hashes 4. Fix some legibility issues
2023-07-07Prevent "spack external find" to error out on wrong permissions (#38755)Massimiliano Culpo2-4/+36
fixes #38733
2023-07-07add info about spack env from spack-configs for oneAPI build tools (#38751)Andrey Parfenov1-0/+47
Signed-off-by: Andrey Parfenov <andrey.parfenov@intel.com>
2023-07-05Drop Python 2 super syntax (#38718)Adam J. Stewart62-245/+199
2023-07-05Drop Python 2 object subclassing (#38720)Adam J. Stewart103-226/+226
2023-07-05Installations: don't set group permissions when they match what is desired ↵Peter Scheibel2-0/+36
(#38036) * When installing a package Spack will attempt to set group permissions on the install prefix even when the configuration does not specify a group. Co-authored-by: David Gomez <dvdgomez@users.noreply.github.com>
2023-07-04spack commands: add type hints and docstrings (#38705)Adam J. Stewart3-159/+371
2023-07-04macos sip: apply on macos only, dont store LD_LIBRARY_PATH (#38687)Harmen Stoppels1-12/+20
2023-07-04spack make-installer: deterministic choice order (#38706)Adam J. Stewart1-3/+2
2023-07-04Remove from __future__ imports (#38703)Adam J. Stewart61-125/+1
2023-07-02SIPPackage: documentation fix (#38672)Adam J. Stewart1-1/+1
2023-06-30gitlab ci: reorganize when we check for specs on mirrors (#38626)Scott Wittenburg1-86/+85
Move the logic checking which mirrors have the specs we need closer to where that information is needed. Also update the staging summary to contain a brief description of why we scheduled or pruned each job. If a spec was found on any mirrors, regardless of whether we scheduled a job for it, print those mirrors.
2023-06-30tests/pythons: convert to new stand-alone test process (#38340)Tamara Dahlgren1-6/+9