summaryrefslogtreecommitdiff
path: root/share
AgeCommit message (Collapse)AuthorFilesLines
2021-02-16Pipelines: Temporary buildcache storage (#21474)Scott Wittenburg1-1/+5
Before this change, in pipeline environments where runners do not have access to persistent shared file-system storage, the only way to pass buildcaches to dependents in later stages was by using the "enable-artifacts-buildcache" flag in the gitlab-ci section of the spack.yaml. This change supports a second mechanism, named "temporary-storage-url-prefix", which can be provided instead of the "enable-artifacts-buildcache" feature, but the two cannot be used at the same time. If this prefix is provided (only "file://" and "s3://" urls are supported), the gitlab "CI_PIPELINE_ID" will be appended to it to create a url for a mirror where pipeline jobs will write buildcache entries for use by jobs in subsequent stages. If this prefix is provided, a cleanup job will be generated to run after all the rebuild jobs have finished that will delete the contents of the temporary mirror. To support this behavior a new mirror sub-command has been added: "spack mirror destroy" which can take either a mirror name or url. This change also fixes a bug in generation of "needs" list for each job. Each jobs "needs" list is supposed to only contain direct dependencies for scheduling purposes, unless "enable-artifacts-buildcache" is specified. Only in that case are the needs lists supposed to contain all transitive dependencies. This changes fixes a bug that caused the needs lists to always contain all transitive dependencies, regardless of whether or not "enable-artifacts-buildcache" was specified.
2021-02-16Add RHEL8 Universal Base Image with platform-python to CI unit tests (#21655)Chuck Atkins1-1/+1
2021-02-16Pipelines: DAG Pruning (#20435)Scott Wittenburg1-2/+6
Pipelines: DAG pruning During the pipeline generation staging process we check each spec against all configured mirrors to determine whether it is up to date on any of the mirrors. By default, and with the --prune-dag argument to "spack ci generate", any spec already up to date on at least one remote mirror is omitted from the generated pipeline. To generate jobs for up to date specs instead of omitting them, use the --no-prune-dag argument. To speed up the pipeline generation process, pass the --check-index-only argument. This will cause spack to check only remote buildcache indices and avoid directly fetching any spec.yaml files from mirrors. The drawback is that if the remote buildcache index is out of date, spec rebuild jobs may be scheduled unnecessarily. This change removes the final-stage-rebuild-index block from gitlab-ci section of spack.yaml. Now rebuilding the buildcache index of the mirror specified in the spack.yaml is the default, unless "rebuild-index: False" is set. Spack assigns the generated rebuild-index job runner attributes from an optional new "service-job-attributes" block, which is also used as the source of runner attributes for another generated non-build job, a no-op job, which spack generates to avoid gitlab errors when DAG pruning results in empty pipelines.
2021-02-12Introduce a SPACK_PYTHON environment variable (#21222)Chuck Atkins3-1/+34
The SPACK_PYTHON environment variable can be set to a python interpreter to be used by the spack command. This allows the spack command itself to use a consistent and separate interpreter from whatever python might be used for package building.
2021-02-09Procedure to deprecate old versions of software (#20767)Adam J. Stewart1-6/+6
* Procedure to deprecate old versions of software * Add documentation * Fix bug in logic * Update tab completion * Deprecate legacy packages * Deprecate old mxnet as well * More explicit docs
2021-02-04spack external find: allow to search by tags (#21407)Massimiliano Culpo1-3/+3
This commit adds an option to the `external find` command that allows it to search by tags. In this way group of executables with common purposes can be grouped under a single name and a simple command can be used to detect all of them. As an example introduce the 'build-tools' tag to search for common development tools on a system
2021-01-27spack setup: remove the command for v0.17.0 (#20277)Adam J. Stewart1-10/+1
spack setup was deprecated in 0.16 and will be removed in 0.17 Follow-up to #18240
2021-01-08Remove ascent gitlab trigger (#20755)Jamie Finney1-9/+0
Remove the ORNL Ascent gitlab trigger CI will now be done internally via periodic builds.
2021-01-05spack python: allow use of IPython (#20329)Vanessasaurus1-1/+1
This adds a -i option to "spack python" which allows use of the IPython interpreter; it can be used with "spack python -i ipython". This assumes it is available in the Python instance used to run Spack (i.e. that you can "import IPython").
2021-01-02copyrights: update all files with license headers for 2021Todd Gamblin23-23/+23
- [x] add `concretize.lp`, `spack.yaml`, etc. to licensed files - [x] update all licensed files to say 2013-2021 using `spack license update-copyright-year` - [x] appease mypy with some additions to package.py that needed for oneapi.py
2021-01-02commands: add `spack license update-copyright-year`Todd Gamblin1-3/+7
This adds a new subcommand to `spack license` that automatically updates the copyright year in files that should have a license header. - [x] add `spack license update-copyright-year` command - [x] add test
2020-12-29PythonPackage: url -> pypi (#20610)Adam J. Stewart1-0/+1
* Convert all `url` attributes in `PythonPackage`s to `pypi` attributes * add `pypi =` to flake8 exceptions
2020-12-22add mypy to style checks; rename `spack flake8` to `spack style` (#20384)Tom Scogland3-4/+17
I lost my mind a bit after getting the completion stuff working and decided to get Mypy working for spack as well. This adds a `.mypy.ini` that checks all of the spack and llnl modules, though not yet packages, and fixes all of the identified missing types and type issues for the spack library. In addition to these changes, this includes: * rename `spack flake8` to `spack style` Aliases flake8 to style, and just runs flake8 as before, but with a warning. The style command runs both `flake8` and `mypy`, in sequence. Added --no-<tool> options to turn off one or the other, they are on by default. Fixed two issues caught by the tools. * stub typing module for python2.x We don't support typing in Spack for python 2.x. To allow 2.x to support `import typing` and `from typing import ...` without a try/except dance to support old versions, this adds a stub module *just* for python 2.x. Doing it this way means we can only reliably use all type hints in python3.7+, and mypi.ini has been updated to reflect that. * add non-default black check to spack style This is a first step to requiring black. It doesn't enforce it by default, but it will check it if requested. Currently enforcing the line length of 79 since that's what flake8 requires, but it's a bit odd for a black formatted project to be quite that narrow. All settings are in the style command since spack has no pyproject.toml and I don't want to add one until more discussion happens. Also re-format `style.py` since it no longer passed the black style check with the new length. * use style check in github action Update the style and docs action to use `spack style`, adding in mypy and black to the action even if it isn't running black right now.
2020-12-22Refactor flake8 handling and tool compatibility (#20376)Tom Scogland2-1/+147
This PR does three related things to try to improve developer tooling quality of life: 1. Adds new options to `.flake8` so it applies the rules of both `.flake8` and `.flake_package` based on paths in the repository. 2. Adds a re-factoring of the `spack flake8` logic into a flake8 plugin so using flake8 directly, or through editor or language server integration, only reports errors that `spack flake8` would. 3. Allows star import of `spack.pkgkit` in packages, since this is now the thing that needs to be imported for completion to work correctly in package files, it's nice to be able to do that. I'm sorely tempted to sed over the whole repository and put `from spack.pkgkit import *` in every package, but at least being allowed to do it on a per-package basis helps. As an example of what the result of this is: ``` ~/Workspace/Projects/spack/spack develop* ⇣ ❯ flake8 --format=pylint ./var/spack/repos/builtin/packages/kripke/package.py ./var/spack/repos/builtin/packages/kripke/package.py:6: [F403] 'from spack.pkgkit import *' used; unable to detect undefined names ./var/spack/repos/builtin/packages/kripke/package.py:25: [E501] line too long (88 > 79 characters) ~/Workspace/Projects/spack/spack refactor-flake8* 1 ❯ flake8 --format=spack ./var/spack/repos/builtin/packages/kripke/package.py ~/Workspace/Projects/spack/spack refactor-flake8* ❯ flake8 ./var/spack/repos/builtin/packages/kripke/package.py ``` * qa/flake8: update .flake8, spack formatter plugin Adds: * Modern flake8 settings for per-path/glob error ignores, allows packages to use the same `.flake8` as the rest of spack * A spack formatter plugin to flake8 that implements the behavior of `spack flake8` for direct invocations. Makes integration with developer tooling nicer, linting with flake8 reports only errors that `spack flake8` would report. Using pyls and pyls-flake8, or any other non-format-dependent flake8 integration, now works with spack's rules. * qa/flake8: allow star import of spack.pkgkit To get working completion of directives and spack components it's necessary to import the contents of spack.pkgkit. At the moment doing this makes flake8 displeased. For now, allow spack.pkgkit and spack both, next step is to ban spack * and require spack.pkgkit *. * first cut at refactoring spack flake8 This version still copies all of the files to be checked as befire, and some other things that probably aren't necessary, but it relies on the spack formatter plugin to implement the ignore logic. * keep flake8 from rejecting itself * remove separate packages flake8 config * fix failures from too many files I ran into this in the PR converting pkgkit to std. The solution in that branch does not work in all cases as it turns out, and all the workarounds I tried to use generated configs to get a single invocation of flake8 with a filename optoion to work failed. It's an astonishingly frustrating config option. Regardless, this removes all temporary file creation from the command and relies on the plugin instead. To work around the huge number of files in spack and still allow the command to control what gets checked, it scans files in batches of 100. This is a completely arbitrary number but was chosen to be safely under common line-length limits. One side-effect of this is that every 100 files the command will produce output, rather than only at the end, which doesn't seem like a terrible thing.
2020-12-18minimal zsh completion (#20253)Tom Scogland5-35/+74
Since zsh can load bash completion files natively, seems reasonable to just turn this on. The only changes are to switch from `type -t` which zsh doesn't support to using `type` with a regex and adding a new arm to the sourcing of the completions to allow it to work for zsh as well as bash. Could use more bash/dash/etc testing probably, but everything I've thought to try has worked so far. Notes: * unit-test zsh support, fix issues Specifically fixed word splitting in completion-test, use a different method to apply sh emulation to zsh loaded bash completion, and fixed an incompatibility in regex operator quoting requirements. * compinit now ignores insecure directories Completion isn't meant to be enabled in non-interactive environments, so by default compinit will ask the user if they want to ignore insecure directories or load them anyway. To pass the spack unit tests in GH actions, this prompt must be disabled, so ignore explicitly until a better solution can be found. * debug functions test also requires bash emulation COMP_WORDS is a bash-ism that zsh doesn't natively support, turn on emulation for just that section of tests to allow the comparison to work. Does not change the behavior of the functions themselves since they are already pinned to sh emulation elsewhere. * propagate change to .in file * fix comment and update script based on .in
2020-12-11Add amazonlinux (x86/arm) dockerfile (#20320)Christian Kniep1-0/+70
Co-authored-by: Christian Kniep <kniec@amazon.com>
2020-12-07Add "spack versions --new" flag to only show new versions (#20030)vvolkl1-1/+1
* [cmd versions] add spack versions --new flag to only fetch new versions format [cmd versions] rename --latest to --newest and add --remote-only [cmd versions] add tests for --remote-only and --new format [cmd versions] update shell tab completion [cmd versions] remove test for --remote-only --new which gives empty output [cmd versions] final rename format * add brillig mock package * add test for spack versions --new * [brillig] format * [versions] increase test coverage * Update lib/spack/spack/cmd/versions.py Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com> * Update lib/spack/spack/cmd/versions.py Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com> Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2020-12-03allow install of build-deps from cache via --include-build-deps switch (#19955)eugeneswalker1-1/+1
* allow install of build-deps from cache via --include-build-deps switch * make clear that --include-build-deps is useful for CI pipeline troubleshooting
2020-12-01dockerfiles: add dockerfile for opensuse leap 15 (#20091)Christian Goll1-0/+61
* added dockerfile for opensuse leap 15 * updated maintainer info * Update share/spack/docker/leap-15.dockerfile * move copies and symlinks after package install also use ${SPACK_ROOT} for spack calls as this works with buildah Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2020-11-20[WIP] nersc e4s pipeline trigger (#19688)Shahzeb Siddiqui1-0/+9
* nersc e4s pipeline trigger * Update nersc_pipeline.yml * Update nersc_pipeline.yml
2020-11-18cmd: add `spack mark` command (#16662)Michael Kuhn1-1/+10
This adds a new `mark` command that can be used to mark packages as either explicitly or implicitly installed. Apart from fixing the package database after installing a dependency manually, it can be used to implement upgrade workflows as outlined in #13385. The following commands demonstrate how the `mark` and `gc` commands can be used to only keep the current version of a package installed: ```console $ spack install pkgA $ spack install pkgB $ git pull # Imagine new versions for pkgA and/or pkgB are introduced $ spack mark -i -a $ spack install pkgA $ spack install pkgB $ spack gc ``` If there is no new version for a package, `install` will simply mark it as explicitly installed and `gc` will not remove it. Co-authored-by: Greg Becker <becker33@llnl.gov>
2020-11-18spack test (#15702)Greg Becker4-7/+101
Users can add test() methods to their packages to run smoke tests on installations with the new `spack test` command (the old `spack test` is now `spack unit-test`). spack test is environment-aware, so you can `spack install` an environment and then run `spack test run` to run smoke tests on all of its packages. Historical test logs can be perused with `spack test results`. Generic smoke tests for MPI implementations, C, C++, and Fortran compilers as well as specific smoke tests for 18 packages. Inside the test method, individual tests can be run separately (and continue to run best-effort after a test failure) using the `run_test` method. The `run_test` method encapsulates finding test executables, running and checking return codes, checking output, and error handling. This handles the following trickier aspects of testing with direct support in Spack's package API: - [x] Caching source or intermediate build files at build time for use at test time. - [x] Test dependencies, - [x] packages that require a compiler for testing (such as library only packages). See the packaging guide for more details on using Spack testing support. Included is support for package.py files for virtual packages. This does not change the Spack interface, but is a major change in internals. Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov> Co-authored-by: wspear <wjspear@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-11-17spack containerize: allow users to customize the base image (#15028)Massimiliano Culpo2-19/+35
This PR reworks a few attributes in the container subsection of spack.yaml to permit the injection of custom base images when generating containers with Spack. In more detail, users can still specify the base operating system and Spack version they want to use: spack: container: images: os: ubuntu:18.04 spack: develop in which case the generated recipe will use one of the Spack images built on Docker Hub for the build stage and the base OS image in the final stage. Alternatively, they can specify explicitly the two base images: spack: container: images: build: spack/ubuntu-bionic:latest final: ubuntu:18.04 and it will be up to them to ensure their consistency. Additional changes: * This commit adds documentation on the two approaches. * Users can now specify OS packages to install (e.g. with apt or yum) prior to the build (previously this was only available for the finalized image). * Handles to avoid an update of the available system packages have been added to the configuration to facilitate the generation of recipes permitting deterministic builds.
2020-11-17concretizer: first working version with pyclingo interfaceTodd Gamblin1-1/+10
- [x] Solver now uses the Python interface to clingo - [x] can extract unsatisfiable cores from problems when things go wrong - [x] use Python callbacks for versions instead of choice rules (this may ultimately hurt performance)
2020-11-16pipelines: support testing PRs from forks (#19248)Scott Wittenburg2-18/+24
This change makes improvements to the `spack ci rebuild` command which supports running gitlab pipelines on PRs from forks. Much of this has to do with making sure we can run without the secrets previously required for running gitlab pipelines (e.g signing key, aws credentials, etc). Specific improvements in this PR: Check if spack has precisely one signing key, and use that information as an additional constraint on whether or not we should attempt to sign the binary package we create. Also, if spack does not have at least one public key, add the install option "--no-check-signature" If we are running a pipeline without any profile or environment variables allowing us to push to S3, the pipeline could still successfully create a buildcache in the artifacts and move on. So just print a message and move on if pushing either the buildcache entry or cdash id file to the remote mirror fails. When we attempt to generate a pacakge or gpg key index on an S3 mirror, and there is nothing to index, just print a warning and exit gracefully rather than throw an exception. Support the use of PR-specific mirrors for temporary binary pkg storage. This will allow quality-of-life improvement for developers, providing a place to store binaries over the lifetime of a PR, so that they must only wait for packages to rebuild from source when they push a new commit that causes it to be necessary. Replace two-pass install with a single pass and the new option: --require-full-hash-match. Doing this also removes the need to save a copy of the spack.yaml to be copied over the one spack rewrites in between the two spack install passes. Work around a mirror configuration issue caused by using spack.util.executable to do the package installation. * Update pipeline trigger jobs for PRs from forks Moving to PRs from forks relies on external synchronization script pushing special branch names. Also secrets will only live on the spack mirror project, and must be propagated to the E4S project via variables on the trigger jobs. When this change is merged, pipelines will not run until we update the "Custom CI configuration path" in the Gitlab CI Settings, as the name of the file has changed to better reflect its purpose. * Arg to MirrorCollection is used exclusively, so add main remote mirror to it * Compute full hash less frequently * Add tests covering index generation error handling code
2020-11-09commands: add `spack tutorial` command (#19808)Todd Gamblin1-1/+5
Added a command to set up Spack for our tutorial at https://spack-tutorial.readthedocs.io. The command does some common operations we need first-time users to do. Specifically: - checks out a particular branch of Spack - deletes spurious configuration in `~/.spack` that might be left over from prior parts of the tutorial - adds a mirror and trusts its public key
2020-10-30Binary caching: use full hashes (#19209)Scott Wittenburg1-1/+1
* "spack install" now has a "--require-full-hash-match" option, which forces Spack to skip an available binary package when the full hash doesn't match. Normally only a DAG-hash match is required, which ensures equivalent Specs, but does not account for changing logic inside the associated package. * Add a local binary cache index which tracks specs that have a binary install available in a remote binary cache. It is updated with "spack buildcache list" or for a given spec when a binary package is retrieved for that Spec.
2020-10-27sbang: use bashcov in sbang on LinuxTodd Gamblin1-2/+7
2020-10-23csh: don't require SPACK_ROOT for sourcing setup-env.csh (#18225)Todd Gamblin5-37/+163
Don't require SPACK_ROOT for sourcing setup-env.csh and make output more consistent
2020-10-21shell support: make `which spack` output intelligible (#19256)Todd Gamblin1-6/+17
Zsh and newer versions of bash have a builtin `which` function that will show you if a command is actually an alias or a function. For functions, the entire function is printed, and our `spack()` function is quite long. Instead of printing out all that, make the `spack()` function a wrapper around `_spack_shell_wrapper()`, and include some no-ops in the definition so that users can see where it was created and where Spack is installed. Here's what the new output looks like in zsh: ```console $ which spack spack () { : this is a shell function from: /Users/gamblin2/src/spack/share/spack/setup-env.sh : the real spack script is here: /Users/gamblin2/src/spack/bin/spack _spack "$@" return $? } ``` Note that `:` is a no-op in Bourne shell; it just discards anything after it on the line. We use it here to embed paths in the function definition (as comments are stripped).
2020-10-18Add testing option to dev-build command (#17293)elsagermann1-1/+1
* ADD: testing to dev-build command * RM: mutally exclusive group for testing in parser * FIX: test option to subparser and not testing * ADD: spack-completion.bash * RM: local devbuildcosmo cmd * FIX: bad merge --drop-in -b --before options forgotten * FIX: --test place in spack-completion.bash * FIX: typo * FIX: blank line removing * FIX: trailing white space Co-authored-by: Elsa Germann <egermann@tsa-ln002.cm.cluster>
2020-10-15Environments: specify packages for developer builds (#15256)Greg Becker1-1/+19
* allow environments to specify dev-build packages * spack develop and spack undevelop commands * never pull dev-build packges from bincache * reinstall dev_specs when code has changed; reinstall dependents too * preserve dev info paths and versions in concretization as special variant * move install overwrite transaction into installer * move dev-build argument handling to package.do_install now that specs are dev-aware, package.do_install can add necessary args (keep_stage=True, use_cache=False) to dev builds. This simplifies driving logic in cmd and env._install * allow 'any' as wildcard for variants * spec: allow anonymous dependencies raise an error when constraining by or normalizing an anonymous dep refactor concretize_develop to remove dev_build variant refactor tests to check for ^dev_path=any instead of +dev_build * fix variant class hierarchy
2020-10-05Revert binary distribution cache manager (#19158)Scott Wittenburg1-1/+1
This reverts #18359 and follow-on PRs intended to address issues with #18359 because that PR changes the hash of all specs. A future PR will reintroduce the changes. * Revert "Fix location in spec.yaml where we look for full_hash (#19132)" * Revert "Fix fetch of spec.yaml files from buildcache (#19101)" * Revert "Merge pull request #18359 from scottwittenburg/add-binary-distribution-cache-manager"
2020-10-02Update buildcache key index when we update the package index (#19117)Scott Wittenburg1-1/+1
This changes makes sure that when we run the pipeline job that updates the buildcache package index on the remote mirror, we also update the key index. The public keys corresponding to the signing keys used to sign the package was pushed to the mirror as a part of creating the buildcache index, so this is just ensuring those keys are reflected in the key index. Also, this change makes sure the "spack buildcache update-index" job runs even when there may have been pipeline failures, since we would like the index always to reflect the true state of the mirror.
2020-09-30Merge pull request #18359 from ↵Scott Wittenburg1-1/+1
scottwittenburg/add-binary-distribution-cache-manager Add binary distribution cache manager
2020-09-25Streamline key management for build caches (#17792)Omar Padron1-1/+10
* Rework spack.util.web.list_url() list_url() now accepts an optional recursive argument (default: False) for controlling whether to only return files within the prefix url or to return all files whose path starts with the prefix url. Allows for the most effecient implementation for the given prefix url scheme. For example, only recursive queries are supported for S3 prefixes, so the returned list is trimmed down if recursive == False, but the native search is returned as-is when recursive == True. Suitable implementations for each case are also used for file system URLs. * Switch to using an explicit index for public keys Switches to maintaining a build cache's keys under build_cache/_pgp. Within this directory is an index.json file listing all the available keys and a <fingerprint>.pub file for each such key. - Adds spack.binary_distribution.generate_key_index() - (re)generates a build cache's key index - Modifies spack.binary_distribution.build_tarball() - if tarball is signed, automatically pushes the key used for signing along with the tarball - if regenerate_index == True, automatically (re)generates the build cache's key index along with the build cache's package index; as in spack.binary_distribution.generate_key_index() - Modifies spack.binary_distribution.get_keys() - a build cache's key index is now used instead of programmatic listing - Adds spack.binary_distribution.push_keys() - publishes keys from Spack's keyring to a given list of mirrors - Adds new spack subcommand: spack gpg publish - publishes keys from Spack's keyring to a given list of mirrors - Modifies spack.util.gpg.Gpg.signing_keys() - Accepts optional positional arguments for filtering the set of keys returned - Adds spack.util.gpg.Gpg.public_keys() - As spack.util.gpg.Gpg.signing_keys(), except public keys are returned - Modifies spack.util.gpg.Gpg.export_keys() - Fixes an issue where GnuPG would prompt for user input if trying to overwrite an existing file - Modifies spack.util.gpg.Gpg.untrust() - Fixes an issue where GnuPG would fail for input that were not key fingerprints - Modifies spack.util.web.url_exists() - Fixes an issue where url_exists() would throw instead of returning False * rework gpg module/fix error with very long GNUPGHOME dir * add a shim for functools.cached_property * handle permission denied error in gpg util * fix tests/make gpgconf optional if no socket dir is available
2020-09-23OLCF Ascent gitlab ci trigger: pass SPACK_REF (#18875)eugeneswalker1-0/+3
2020-09-18trigger ascent e4s pipeline on merge to spack develop (#18655)Shahzeb Siddiqui1-0/+6
* trigger ascent e4s pipeline on merge to spack develop * change pipeline name ecpcitest/e4s is the pipeline that will be triggered for merge on develop its the E4S use-case.
2020-09-14Make sure each develop pipeline tests associated commitScott Wittenburg1-1/+1
2020-09-14Provide your own script, before_script, and after_scriptScott Wittenburg1-1/+1
2020-09-10Bugfix for fish support: overly zealous arg matching (#18528)Johannes Blaschke1-18/+59
* bugfix for issue 18369 * fix typo Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-09-08commands: update help for `spack install --yes-to-all` (#18367)Richarda Butler1-1/+1
`spack install --yes-to-all` doesn't actually make the build non-interactive, but that is why people typically use it. This documents that you must also specify `--no-checksum` for a fully non-interactive build.
2020-09-05Make spack environment configurations writable from spack external and spack ↵Robert Blake1-1/+1
compiler find (#18165) * spack config: default modification scope can be an environment The previous model was that environments are the highest priority config scope for config reading operations, but were not considered for config writing operations. Now, the active environment is the highest priority config scope for both reading and writing operations. Now spack config add, spack external find and spack compiler set environment configuration in the environment by default if an environment is active. This is a change in default behavior for these routines, but better matches the mental model for an environment taking precedence over the user's default config file. * add scope argument to 'spack external find' to choose non-default scope * Increase testing for config modifications on environments Co-authored-by: Gregory Becker <becker33@llnl.gov>
2020-09-04Rely on E4S project variable for SPACK_REPOScott Wittenburg1-2/+0
2020-09-02Add new RubyPackage build system base class (#18199)Adam J. Stewart1-1/+5
* Add new RubyPackage build system base class * Ruby: add spack external find support * Add build tests for RubyPackage
2020-08-10Simplify the detection protocol for packagesMassimiliano Culpo1-1/+5
Packages can implement “detect_version” to support detection of external instances of a package. This is generally easier than implementing “determine_spec_details”. The API for determine_version is similar: for example you can return “None” to indicate that an executable is not an instance of a package. Users may implement a “determine_variants” method for a package. When doing external detection, executables are grouped by version and each group results in a single invocation of “determine_variants” for the associated spec. The method returns a string specifying the variants for the package. The method may additionally return a dictionary representing extra attributes for the package. These will be stored in the spec yaml and can be retrieved from self.spec.extra_attributes The Spack GCC package has been updated with an implementation of “determine_variants” which adds the following extra attributes to the package: c, cxx, fortran
2020-08-10Update packages.yaml format and support configuration updatesMassimiliano Culpo2-14/+56
The YAML config for paths and modules of external packages has changed: the new format allows a single spec to load multiple modules. Spack will automatically convert from the old format when reading the configs (the updates do not add new essential properties, so this change in Spack is backwards-compatible). With this update, Spack cannot modify existing configs/environments without updating them (e.g. “spack config add” will fail if the configuration is in a format that predates this PR). The user is prompted to do this explicitly and commands are provided. All config scopes can be updated at once. Each environment must be updated one at a time.
2020-07-31Move Python 2.6 unit tests to Github Actions (#17279)Massimiliano Culpo1-22/+0
* Run Python2.6 unit tests on Github Actions * Skip url tests on Python 2.6 to reduce waiting times * Skip foreground background tests on Python 2.6 to reduce waiting times * Removed references to Travis in the documentation * Deleted install_patchelf.sh (can be installed from repo on CentOS 6)
2020-07-27tutorial: Add boto3 installation to setup script (#17722)Todd Gamblin1-0/+6
2020-07-27add tutorial setup script to share/spack (#17705)Greg Becker1-0/+123
* add tutorial setup script to share/spack * Add check for Ubuntu 18, fix xvda check, fix apt-get errors - now works on t2.micro, t2.small, and m instances - apt-get needs retries around it to work Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>