summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2022-07-31black: configurationTodd Gamblin2-39/+56
This adds necessary configuration for flake8 and black to work together. This also sets the line length to 99, per the data here: * https://github.com/spack/spack/pull/24718#issuecomment-876933636 Given the data and the spirit of black's 88-character limit, we set the limit to 99 characters for all of Spack, because: * 99 is one less than 100, a nice round number, and all lines will fit in a 100-character wide terminal (even when the text editor puts a \ at EOL). * 99 is just past the knee the file size curve for packages, and it means that packages remain readable and not significantly longer than they are now. * It doesn't seem to hurt core -- files in core might change length by a few percent but seem like they'll be mostly the same as before -- just a bit more roomy. - [x] set line length to 99 - [x] remove most exceptions from `.flake8` and add the ones black cares about - [x] add `[tool.black]` to `pyproject.toml` - [x] make `black` run if available in `spack style --fix` Co-Authored-By: Tom Scogland <tscogland@llnl.gov>
2022-07-30MACOSX_DEPLOYMENT_TARGET: MAJOR.MINOR (#31811)Adam J. Stewart1-1/+2
2022-07-28environment.py: only install root specs explicitly (#31645)Harmen Stoppels2-3/+6
2022-07-28Fix parsing of clean_url (#31783)Stephen Sachs1-2/+2
In #31618 the idea was to determine the file extension heuristically by dropping query params etc from a url and then consider it as a file path. That broke for URLs that only have query params like http://example.com/?patch=x as it would result in empty string as basename. This PR reverts to the old behavior of saving files as ?patch=x in that case. Co-authored-by: Stephen Sachs <stesachs@amazon.com>
2022-07-26environment.py: reduce # of locks further (#31643)Harmen Stoppels2-27/+49
* environment.py: reduce # of locks further
2022-07-26containerize: fix concretization -> concretizer (#31594)Harmen Stoppels3-8/+8
* containerize: fix concretization -> concretizer * fix test
2022-07-26llvm: Variant and dependency cleanup (#31331)Chuck Atkins2-111/+171
* llvm: Use variant when clauses for many of the expressed conflicts * llvm: Remove the shared variant as it wasn't really used * llvm: Remove unnecessary deps and make explicit the ones that are * llvm: Cleanup patch conditions * pocl: Update for llvm cleanup * unit-test: update unparse package hash with the updated llvm package * llvm: Fix ppc long double patching and add clarifying comments
2022-07-26Fix `self.extension` for symlinks (#31743)Stephen Sachs1-1/+3
`self.archive_file` is (among others) a symlink to a tarball. `extension()` on a symlink will result in no extension. This patch fixes the behavior introduced in https://github.com/spack/spack/pull/31618. Co-authored-by: Stephen Sachs <stesachs@amazon.com>
2022-07-25Fix stage with uri params (#31618)Harmen Stoppels2-3/+3
* don't determine extension of a url * parse url to drop query params from filename
2022-07-24Sphinx 5.1.0 breaks doc builds (#31709)Adam J. Stewart1-1/+1
2022-07-22Mark external as explicit only when installing explicitly (#31665)Harmen Stoppels2-5/+18
2022-07-22autotools: add -I flag when non-standard libtool (#31677)Harmen Stoppels2-6/+99
When 1. Spack installs libtool, 2. system libtool is installed too, and 3. system automake is used Spack passes system automake's `-I <prefix>` flag to itself, even though it's a default search path. This takes precedence over spack's libtool prefix dir. This causes the wrong `libtool.m4` file to be used (since system libtool is in the same prefix as system automake). And that leads to error messages about incompatible libtool, something something LT_INIT.
2022-07-22Use pkg_cls in spack.mirror.get_all_versions (#31636)Massimiliano Culpo2-7/+22
fixes #31627 spack.mirror.get_all_versions now uses the package class instead of the package object in its implementation. Ensure spec is concrete before staging for mirrors
2022-07-20Don't restage develop specs when a patch fails (#31593)psakievich2-3/+55
* make develop specs not restage when a patch fails * add a unit test
2022-07-20spack stage: add missing --fresh and --reuse (#31626)Harmen Stoppels2-0/+12
2022-07-20installer.py: require "explicit: True" in the install arguments to mark a ↵Harmen Stoppels2-1/+14
package "explicit" (#31646)
2022-07-18Only hack botocore when needed (#31610)Jonathon Anderson1-6/+10
Newer versions of botocore (>=1.23.47) support the full IOBase interface, so the hacks added to supplement the missing attributes are no longer needed. Conditionally disable the hacks if they appear to be unnecessary based on the class hierarchy found at runtime.
2022-07-15Use lexists instead of exists during fetch (#31509)Harmen Stoppels1-11/+11
2022-07-14Bug/fix credentials s3 buildcache update (#31391)Joseph Snyder3-5/+14
* Add connection information to buildcache update command Ensure that the s3 connection made when attempting to update the content of a buildcache attempts to use the extra connection information from the mirror creation. * Add unique help for endpoint URL argument Fix copy/paste error for endpoint URL help which was the same as the access token * Re-work URL checking for S3 mirrors Due to the fact that nested bucket URLs would never match the string used for checking that the mirror is the same, switch the check used. Sort all mirror URLs by length to have the most specific cases first and see if the desired URL "starts with" the mirror URL. * Long line style fixes Add execptions for long lines and fix other style errors * Use format() function to rebuild URL Use the format command to rebuild the url instead of crafing a formatted string out of known values * Add early exit for URL checking When a valid mirror is found, break from the loop
2022-07-14Deprecate `blacklist`/`whitelist` in favor of `include`/`exclude` (#31569)Todd Gamblin23-135/+375
For a long time the module configuration has had a few settings that use `blacklist`/`whitelist` terminology. We've been asked by some of our users to replace this with more inclusive language. In addition to being non-inclusive, `blacklist` and `whitelist` are inconsistent with the rest of Spack, which uses `include` and `exclude` for the same concepts. - [x] Deprecate `blacklist`, `whitelist`, `blacklist_implicits` and `environment_blacklist` in favor of `exclude`, `include`, `exclude_implicits` and `exclude_env_vars` in module configuration, to be removed in Spack v0.20. - [x] Print deprecation warnings if any of the deprecated names are in module config. - [x] Update tests to test old and new names. - [x] Update docs. - [x] Update `spack config update` to fix this automatically, and include a note in the error that you can use this command.
2022-07-13Decompression: use tar exe vs. built-in Python tarfile support (#31563)Peter Scheibel1-19/+4
Python's built-in tarfile support doesn't address some general cases of malformed tarfiles that are already handled by the system 'tar' utility; until these can be addressed, use that exclusively.
2022-07-12`spack.repo.get()` can only be called on concrete specs (#31411)Massimiliano Culpo76-1397/+1065
The goal of this PR is to make clearer where we need a package object in Spack as opposed to a package class. We currently instantiate a lot of package objects when we could make do with a class. We should use the class when we only need metadata, and we should only instantiate and us an instance of `PackageBase` at build time. Modifications: - [x] Remove the `spack.repo.get` convenience function (which was used in many places, and not really needed) - [x] Use `spack.repo.path.get_pkg_class` wherever possible - [x] Try to route most of the need for `spack.repo.path.get` through `Spec.package` - [x] Introduce a non-data descriptor, that can be used as a decorator, to have "class level properties" - [x] Refactor unit tests that had to be modified to reduce code duplication - [x] `Spec.package` and `Repo.get` now require a concrete spec as input - [x] Remove `RepoPath.all_packages` and `Repo.all_packages`
2022-07-12file_cache.py: idempotent remove without races (#31477)Harmen Stoppels2-4/+19
There's a race condition in `remove()` as the lockfile is removed after releasing the lock, which is a problem when another process acquires a write lock during deletion. Also simplify life a bit in multiprocessing when a file is possibly removed multiple times, which currently is an error on the second deletion, so the proposed fix is to make remove(...) idempotent and not error when deleting non-existing cache entries. Don't tests for existence of lockfile, cause windows/linux behavior is different
2022-07-12compression.py: buffered copy (#31533)Harmen Stoppels1-3/+4
2022-07-12environment.py: only acquire write lock when necessary (#31493)Harmen Stoppels1-3/+4
2022-07-12env depfile: add missing touch (#31494)Harmen Stoppels1-0/+1
Oversight in #31433, the non-phony `env` target was missing a file being created for it, which can cause make to infinitely loop when including multiple generated makefiles.
2022-07-12Remove os.getcwd from function signature (#31480)Massimiliano Culpo1-3/+2
fixes #29730 This is also wrong since it binds the directory at the time of module import, rather than the one at the time of function call.
2022-07-12Sanitize ownership when extracting tarfiles (#31524)Jonathon Anderson1-3/+9
2022-07-11py-hatchling: add new packages, document new build backend (#31512)Adam J. Stewart1-2/+31
* py-hatchling: add new package, document new build backend * Minor doc changes Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com> Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2022-07-07Fix spack edit message when no editor installed (#31412)Cory Bloor2-1/+15
When no default editor is installed and no environment variable is set, which_string would return None and this would be passed to os.execv resulting in a TypeError. The message presented to the user would be: Error: execv: path should be string, bytes or os.PathLike, not NoneType This change checks that which_string has returned successfully before attempting to execute the result, resulting in a new error message: Error: No text editor found! Please set the VISUAL and/or EDITOR environment variable(s) to your preferred text editor. It's not strictly necessary, but I've also changed try_exec to catch all errors rather than just OSErrors. This would have provided slightly more context for the original error message.
2022-07-07Documentation: Add SourceforgePackage to the build systems docs (#31473)Tamara Dahlgren2-2/+58
2022-07-07removing feature bloat: monitor and analyzers (#31130)Vanessasaurus18-2159/+0
Signed-off-by: vsoch <vsoch@users.noreply.github.com> Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2022-07-06Testing Bugfix: refactor clean --python-cache to support all (#31449)Tamara Dahlgren2-13/+63
There were two choices: 1) remove '-p' from '-a' or 2) allow monkeypatching the cleaning of the python cache since clean's test_function_calls isn't supposed to be actually cleaning anything. This commit supports the latter and adds a test case for `-p`.
2022-07-05Remove fetch from depfile (#31433)Harmen Stoppels3-51/+19
2022-07-05ROCmPackage: unify amdgpu_targets (#30582)Alberto Invernizzi1-3/+4
* unify amdgpu_targets for rocsolver and rocblas * add more archs * adapt all roc packages to unified amdgpu_targets
2022-07-01gitlab: Fix pipeline generation for non-develop protected pipelines (#31378)Scott Wittenburg1-6/+9
Release branches and tags run protected pipelines, and we noticed that those pipelines were generating all jobs in the stack, even when the mirror contained all the built specs and an up to date index. The problem was caused because the override mirror was not present in spacks mirror configuration at the time when the binary_distribution.update() method was called. This fixes that by always adding the mirror override, if present, to the list of configured mirrors.
2022-06-29Cray manifest: compiler duplicates (#31173)Peter Scheibel3-7/+33
* remove unhelpful comment * Filter compiler duplicates while reading manifest * more-specific version matching edited to use module-specific version (to avoid an issue where a user might add a compiler with the same version to the initial test configuration
2022-06-29PythonPackage: add default libs/headers attributes (#28527)Adam J. Stewart1-0/+34
* PythonPackage: add default libs/headers attributes * Style fix * libs and headers should be properties * Check both platlib and include * Fix variable reference
2022-06-29Update containerize templates to account for view indirection (#31321)Massimiliano Culpo1-1/+2
fixes #30965
2022-06-28oneAPI supports cxx17_flag and cxx20_flag (#31274)Wouter Deconinck1-0/+8
2022-06-28Feature: use git branches/tags as versions (#31200)Greg Becker8-120/+304
Building on #24639, this allows versions to be prefixed by `git.`. If a version begins `git.`, it is treated as a git ref, and handled as git commits are starting in the referenced PR. An exception is made for versions that are `git.develop`, `git.main`, `git.master`, `git.head`, or `git.trunk`. Those are assumed to be greater than all other versions, as those prefixed strings are in other contexts.
2022-06-27oneapi-* packages: improve use with modules (#30981)Sergey Kosukhin1-14/+12
This commit adds some changes which improve use of Spack-installed oneAPI packages with Spack-generated modules, but not within Spack (e.g. if you install some of these packages with Spack, then load their associated modules to build other packages outside of Spack). The majority of the PR diff is refactoring. The functional changes are: * intel-oneapi-mkl: * setup_run_environment: update Intel compiler flags to RPATH the mkl libs * intel-oneapi-compilers: update the compiler configuration to RPATH libraries installed by this package (note that Spack already handled this when installing dependent packages with Spack, but this is specifically to use the intel-oneapi-compilers package outside of Spack). Specifically: * inject_rpaths: this modifies the binaries installed as part of the intel-oneapi-compilers package to RPATH libraries provided by this package * extend_config_flags: this instructs the compiler executables provided by this package to RPATH those same libraries Refactoring includes: * intel-oneapi-compilers: in addition to the functional changes, a large portion of this diff is reorganizing the creation of resources/archives downloaded for the install * The base oneAPI package renamed component_path, so several packages changed as a result: * intel-oneapi-dpl * intel-oneapi-dnn * extrae * intel-oneapi-mpi: * Perform file filtering in one pass rather than two
2022-06-27`spack external find`: handle manifest with bad permissions (#31201)Peter Scheibel4-0/+98
Allow `spack external find` (with no extra args) to proceed if the manifest file exists but without sufficient permissions; in that case, print a warning. Also add a test for that behavior. TODOs: - [x] continue past any exception raised during manifest parsing as part of `spack external find`, except for CTRL-C (and other errors that indicate immediate program termination) - [x] Semi-unrelated but came up when discussing this with the user who reported this issue to me: the manifest parser now accepts older schemas See: https://github.com/spack/spack/issues/31191
2022-06-27Rename os_facts.lp to os_compatibility.lp (#31206)Massimiliano Culpo2-1/+1
Follow-up from #31170
2022-06-24Fix typo in documentation note about concretizer:unify (#31246)Cody Balos1-3/+3
2022-06-23ASP-based solver: rescale target weights so that 0 is always the best score ↵Massimiliano Culpo2-49/+15
(#31226) fixes #30997 Instead of giving a penalty of 30 to all nodes when preferences are not package specific, give a penalty of 100 to all targets of a node where we have package specific preferences, if the target is not explicitly preferred.
2022-06-22relocation: x-pie-executable needs relocation (#31248)eugeneswalker1-1/+2
2022-06-22spack create: fix for no URL (#31239)Adam J. Stewart2-1/+7
2022-06-22Remove update functions used to ease the transition in v0.16 (#31216)Massimiliano Culpo4-301/+12
2022-06-21Canonicalize positional argument to spack bootstrap mirror (#31180)Massimiliano Culpo1-1/+3
fixes #31139