summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2020-10-11Add testing for Python 3.9 (#19261)Adam J. Stewart1-5/+5
2020-10-11CUDA: update maintainers (#19262)Adam J. Stewart1-1/+1
2020-10-09Use https for links (#19244)谭九鼎2-4/+4
2020-10-05Revert binary distribution cache manager (#19158)Scott Wittenburg11-623/+139
This reverts #18359 and follow-on PRs intended to address issues with #18359 because that PR changes the hash of all specs. A future PR will reintroduce the changes. * Revert "Fix location in spec.yaml where we look for full_hash (#19132)" * Revert "Fix fetch of spec.yaml files from buildcache (#19101)" * Revert "Merge pull request #18359 from scottwittenburg/add-binary-distribution-cache-manager"
2020-10-02Fix location in spec.yaml where we look for full_hash (#19132)Scott Wittenburg2-8/+63
When we attempt to determine whether a remote spec (in a binary mirror) is up-to-date or needs to be rebuilt, we compare the full_hash stored in the remote spec.yaml file against the full_hash computed from the local concrete spec. Since the full_hash moved into the spec (and is no longer at the top level of the spec.yaml), we need to look there for it. This oversight from #18359 was causing all specs to get rebuilt when the full_hash wasn't fouhd at the expected location.
2020-10-02Update buildcache key index when we update the package index (#19117)Scott Wittenburg4-2/+58
This changes makes sure that when we run the pipeline job that updates the buildcache package index on the remote mirror, we also update the key index. The public keys corresponding to the signing keys used to sign the package was pushed to the mirror as a part of creating the buildcache index, so this is just ensuring those keys are reflected in the key index. Also, this change makes sure the "spack buildcache update-index" job runs even when there may have been pipeline failures, since we would like the index always to reflect the true state of the mirror.
2020-10-01Fix fetch of spec.yaml files from buildcache (#19101)Scott Wittenburg1-6/+3
Since those files currently exist in buildcaches (in S3 buckets) with potentially different content types, we should be less restrictive in what content types we accept when attempting to fetch them. This PR removes the content type constraint so any file with the matching name will be found.
2020-10-01Binary caching: avoid duplicate RPATHs, unnecessary updates (#19061)Patrick Gartung1-5/+8
* Remove duplication of reconstructed RPATHs caused by multiple identical entries in prefixes dictionary * Don't rewrite RPATHs if relative RPATHs are unchanged because the directory layout is unchanged
2020-10-01Buildcache: Need to check the binary is not a Mach-o binary in a linux ↵Patrick Gartung1-1/+6
package or an ELF binary in a macOS package. (#18670) * Need to check the binary is not a Mach-o binary in a linux package or an ELF binary in a macOS package. * use sys.platform * Darwin -> darwin for sys.platform
2020-09-30Merge pull request #18359 from ↵Scott Wittenburg11-131/+563
scottwittenburg/add-binary-distribution-cache-manager Add binary distribution cache manager
2020-09-29CUDA: added v11.1.0 (#19036)Axel Huebl1-6/+14
Compiler conflicts have been updated accordingly
2020-09-28Fixup conflicts for CUDA 11.0.2 and GCC (#19035)David Beckingsale1-2/+4
* Fixup conflicts for CUDA 11.0.2 and GCC * Updates for ppc64le * Fix missing "or newer" Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
2020-09-28autotools: patch 'libtool' recursively in subdirectories (#18620)Tomoki, Karatsu1-17/+20
Previous version was doing it only in the root build directory.
2020-09-28Fix usage of builtin file as variable name (#19018)Adam J. Stewart2-34/+35
2020-09-25Streamline key management for build caches (#17792)Omar Padron13-225/+736
* Rework spack.util.web.list_url() list_url() now accepts an optional recursive argument (default: False) for controlling whether to only return files within the prefix url or to return all files whose path starts with the prefix url. Allows for the most effecient implementation for the given prefix url scheme. For example, only recursive queries are supported for S3 prefixes, so the returned list is trimmed down if recursive == False, but the native search is returned as-is when recursive == True. Suitable implementations for each case are also used for file system URLs. * Switch to using an explicit index for public keys Switches to maintaining a build cache's keys under build_cache/_pgp. Within this directory is an index.json file listing all the available keys and a <fingerprint>.pub file for each such key. - Adds spack.binary_distribution.generate_key_index() - (re)generates a build cache's key index - Modifies spack.binary_distribution.build_tarball() - if tarball is signed, automatically pushes the key used for signing along with the tarball - if regenerate_index == True, automatically (re)generates the build cache's key index along with the build cache's package index; as in spack.binary_distribution.generate_key_index() - Modifies spack.binary_distribution.get_keys() - a build cache's key index is now used instead of programmatic listing - Adds spack.binary_distribution.push_keys() - publishes keys from Spack's keyring to a given list of mirrors - Adds new spack subcommand: spack gpg publish - publishes keys from Spack's keyring to a given list of mirrors - Modifies spack.util.gpg.Gpg.signing_keys() - Accepts optional positional arguments for filtering the set of keys returned - Adds spack.util.gpg.Gpg.public_keys() - As spack.util.gpg.Gpg.signing_keys(), except public keys are returned - Modifies spack.util.gpg.Gpg.export_keys() - Fixes an issue where GnuPG would prompt for user input if trying to overwrite an existing file - Modifies spack.util.gpg.Gpg.untrust() - Fixes an issue where GnuPG would fail for input that were not key fingerprints - Modifies spack.util.web.url_exists() - Fixes an issue where url_exists() would throw instead of returning False * rework gpg module/fix error with very long GNUPGHOME dir * add a shim for functools.cached_property * handle permission denied error in gpg util * fix tests/make gpgconf optional if no socket dir is available
2020-09-25refactor install_tree to use projections format (#18341)Greg Becker10-97/+239
* refactor install_tree to use projections format * Add update method for config.yaml * add test for config update config
2020-09-21typo (#18845)Greg Becker1-1/+1
2020-09-19Ensure variant defaults are parsable from CLI. (#18661)Massimiliano Culpo1-1/+17
- Add a unit test to check if there are unparsable defaults - Fix 'rust' and 'nsimd' variants
2020-09-18do out of source builds in hashed directories (#18574)Greg Becker1-1/+2
2020-09-18env view failures: print underlying error message (#18713)Greg Becker2-1/+13
2020-09-14Do not assume we sit in the directory where the env file lives.Scott Wittenburg1-3/+8
2020-09-14Remove :<name> interpolation, add SPACK_VERSION variablesScott Wittenburg3-128/+94
Also fix issues with documentation to reflect changes
2020-09-14Document pipeline keys which can be global but overriddenScott Wittenburg1-6/+108
Update pipelines documentation to describe how 'tags', 'variables', 'image', 'before_script', 'script', and 'after_script' can be supplied at the top level, to be used by any of the runner mappings, and also overridden by any of the runner mappings. Also show an example of capturing the custom spack SHA at pipeline generation time, so all jobs are sure to run with the same version of spack, as a means to illustrate the $env:VARIABLE_NAME syntax.
2020-09-14Allow overridable global runner attributesScott Wittenburg3-11/+205
2020-09-14Update pipeline documentation to describe user-provided scriptsScott Wittenburg1-17/+73
2020-09-14Add test for variable interpolation and scriptsScott Wittenburg1-0/+82
2020-09-14Support variable interpolation at pipeline generation timeScott Wittenburg1-0/+10
2020-09-14Provide your own script, before_script, and after_scriptScott Wittenburg4-111/+34
2020-09-14Fix a typo in test/concretize.pyi (#18662)Massimiliano Culpo1-1/+1
2020-09-11bugfix: include configuration ignoring files with the same basename (#18487)Richarda Butler2-2/+43
* Use the config path instead of the basename * Removing unused variables Co-authored-by: Greg Becker <becker33@llnl.gov> * Test Making sure if there are 2 include config files with the same basename they are both implemented * Edit test assert Co-authored-by: Greg Becker <becker33@llnl.gov>
2020-09-11Environments: Avoid inconsistent state on failed write (#18538)Massimiliano Culpo4-19/+206
Fixes #18441 When writing an environment, there are cases where the lock file for the environment may be removed. In this case there was a period between removing the lock file and writing the new manifest file where an exception could leave the manifest in its old state (in which case the lock and manifest would be out of sync). This adds a context manager which is used to restore the prior lock file state in cases where the manifest file cannot be written.
2020-09-09tcl module files: fix configuration overriding (#18514)Rémi Lacroix3-5/+28
This is a special case of overriding since each section is being matched with the current spec. The trailing ':' for sections with override is now removed when parsing the configuration so the special handling for the modules configuration stopped working but it went unnoticed.
2020-09-08Clarify manual download required if unable to fetch package (#18242)Tamara Dahlgren4-11/+87
Clarify manual download required if unable to fetch (from mirror(s)); support (and tests) for package-specific download instructions
2020-09-08commands: update help for `spack install --yes-to-all` (#18367)Richarda Butler1-1/+8
`spack install --yes-to-all` doesn't actually make the build non-interactive, but that is why people typically use it. This documents that you must also specify `--no-checksum` for a fully non-interactive build.
2020-09-08print out debug information about which specs are applying which constraintsPeter Josef Scheibel1-0/+2
2020-09-08spack docs: http -> https (#18573)Adam J. Stewart1-1/+1
2020-09-08Modules: Deduplicate suffixes but don't sort them. (#18351)Rémi Lacroix4-5/+11
* Modules: Deduplicate suffixes but don't sort them. The suffixes' order is defined by the order in which they appear in the configuration file. * Modules: Modify tests to use spack_yaml.load_config. spack_yaml.load_config ensures that the configuration is stored in an ordered manner. Without this change, the behavior of the tests did not match Spack's. * Modules: Tweak the suffixes test to better catch ordering issues.
2020-09-07SpecList: remove mutable types from __init__ arguments (#18515)Massimiliano Culpo1-1/+5
fixes #18439
2020-09-05Make spack environment configurations writable from spack external and spack ↵Robert Blake7-56/+125
compiler find (#18165) * spack config: default modification scope can be an environment The previous model was that environments are the highest priority config scope for config reading operations, but were not considered for config writing operations. Now, the active environment is the highest priority config scope for both reading and writing operations. Now spack config add, spack external find and spack compiler set environment configuration in the environment by default if an environment is active. This is a change in default behavior for these routines, but better matches the mental model for an environment taking precedence over the user's default config file. * add scope argument to 'spack external find' to choose non-default scope * Increase testing for config modifications on environments Co-authored-by: Gregory Becker <becker33@llnl.gov>
2020-09-03depends_on cannot handle ^ sigil (#18220)Adam J. Stewart1-1/+28
* depends_on cannot handle ^ sigil * cardioid+mfem+cuda requires hypre+cuda * Document this limitation * Move warning message to Known Issues docs * Better handling of parmetis dep
2020-09-03Add new MavenPackage build system base class (#18185)Adam J. Stewart6-0/+155
* Add new MavenPackage build system base class * Fix flake8 and doc tests * More specific regex * Java 8 required for these packages
2020-09-03Hashing: force hash consistency for values read from config (#18446)Massimiliano Culpo2-3/+56
The 'external_modules' attribute on a Spec, when read from a YAML configuration file, may contain extra formatting that is lost when that Spec is written-to/read-from JSON format. This was resulting in a hashing instability (when the Spec was read back, it would report a different hash). This commit adds a function which removes the extra formatting from 'external_modules' as it is passed to the Spec in __init__ to ensure a consistent hash.
2020-09-03install/install_tree: glob support (#18376)Adam J. Stewart2-64/+200
* install/install_tree: glob support * Add unit tests * Update existing packages * Raise error if glob finds no files, document function raises
2020-09-03Bugfix: terminate if a spack.yaml include path does not exist (#18074)Tamara Dahlgren2-1/+32
2020-09-02Add new RubyPackage build system base class (#18199)Adam J. Stewart5-8/+275
* Add new RubyPackage build system base class * Ruby: add spack external find support * Add build tests for RubyPackage
2020-09-02PythonPackage: update documentation (#18181)Adam J. Stewart2-36/+195
2020-09-02spack test: no gpg signing for git commits (#18454)Adam J. Stewart2-7/+11
2020-09-02Document test dependency type (#18365)Adam J. Stewart1-10/+18
2020-09-02Mac OS: support Python >= 3.8 by using fork-based multiprocessing (#18124)Rui Xue6-8/+29
As detailed in https://bugs.python.org/issue33725, starting new processes with 'fork' on Mac OS is not guaranteed to work in general. As of Python 3.8 the default process spawning mechanism was changed to avoid this issue. Spack depends on the fork-based method to preserve file descriptors transparently, to preserve global state, and to avoid pickling some objects. An effort is underway to remove dependence on fork-based process spawning (see #18205). In the meantime, this allows Spack to run with Python 3.8 on Mac OS by explicitly choosing to use 'fork'. Co-authored-by: Peter Josef Scheibel <scheibel1@llnl.gov> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2020-09-01Deprecate spack setup (#18240)Adam J. Stewart6-526/+5
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2024-11-01 18:45:28 +0000