summaryrefslogtreecommitdiff
path: root/share
AgeCommit message (Collapse)AuthorFilesLines
2019-10-21Bootstrap environment modules optimizing for generic architectures (#13105)Massimiliano Culpo1-0/+37
fixes #13073 Since #3206 was merged bootstrapping environment-modules was using the architecture of the current host or the best match supported by the default compiler. The former case is an issue since shell integration was looking for a spec targeted at the host microarchitecture. 1. Bootstrap an env modules targeted at generic architectures 2. Look for generic targets in shell integration scripts 3. Add a new entry in Travis to test shell integration
2019-10-15lmod: module files are written in a root folder named by target family (#13121)Massimiliano Culpo1-0/+1
fixes #13005 This commit fixes an issue with the name of the root directory for module file hierarchies. Since #3206 the root folder was named after the microarchitecture used for the spec, which is too specific and not backward compatible for lmod hierarchies. Here we compute the root folder name using the target family instead of the target name itself and we add target information in the 'whatis' portion of the module file.
2019-10-02Remove support for generating dotkit files (#11986)Massimiliano Culpo6-166/+25
Dotkit is being used only at a few sites and has been deprecated on new machines. This commit removes all the code that provide support for the generation of dotkit module files. A new validator named "deprecatedProperties" has been added to the jsonschema validators. It permits to prompt a warning message or exit with an error if a property that has been marked as deprecated is encountered. * Removed references to dotkit in the docs * Removed references to dotkit in setup-env-test.sh * Added a unit test for the 'deprecatedProperties' schema validator
2019-10-01Add all compatible system types directory to module pathsMassimiliano Culpo2-2/+7
fixes #12915 closes #12916 Since Spack has support for specific targets it might happen that software is built for targets that are not exactly the host because it was either an explicit user request or the compiler being used is too old to support the host. Modules for different targets are written into different directories and by default Spack was adding to MODULEPATH only the directory corresponding to the current host. This PR modifies this behavior to add all the directories that are **compatible** with the current host.
2019-09-20Add --known-targets to bash completion for arch command (#12887)Glenn Johnson1-1/+1
This PR adds the new --known-targets flag to the `spack arch` command.
2019-09-13Refactor release-jobs cmd based on use of environments (no docker either)Scott Wittenburg1-11/+0
2019-09-12BUG: recursively load modules with tcsh (#12664)Tyler Reddy1-1/+6
* for tcsh and csh, spack load -r package should now correctly load recursively instead of only loading the target package without any dependencies
2019-09-09packages service: fix docker build again (#12774)Axel Huebl1-1/+1
CD is hard.
2019-09-09packages service: fix docker build (#12773)Axel Huebl1-1/+1
The build instructions I cloned from did not work ;)
2019-09-09packages build: fix path to spack (#12772)Axel Huebl1-1/+4
Fix docker build for packages.spack.io
2019-09-09Fix CD: Packages Service First (#12764)Axel Huebl1-81/+2
* Fix CD: Packages Service First Build the packages.spack.io service images first, so they are guaranteed to be pushed even if further images fail to build. Fix the query to the `spack` script executed in later builds. * CD: Remove Spack Images Now done on Dockerhub.
2019-09-08Add CORS for spack.packages.ioAxel Huebl2-0/+4
Add the HTTP header `Access-Control-Allow-Origin: *` for our NGINX service that is serving static JSON content on https://spack.packages.io .
2019-09-02tests and completions for `spack find --json` and `spack find --format`Todd Gamblin1-2/+3
2019-07-24new command: `spack maintainers` queries package maintainersTodd Gamblin1-0/+10
- We don't currently make enough use of the maintainers field on packages, though we could use it to assign reviews. - add a command that allows maintainers to be queried - can ask who is maintaining a package or packages - can ask what packages users are maintaining - can list all maintained or unmaintained packages - add tests for the command
2019-07-20docker: fix builds, remove extra distros, add ci builds (#11621)Omar Padron26-643/+383
* fix docker builds/remove extra builds/add ci builds * preinstall vim in CI builder images * simplify & streamline docker resources * restore os-container-mapping.yaml file
2019-07-20environments: add activate/deactivate tests, work wtih `set -u`Todd Gamblin1-9/+89
- [x] Add shell tests to ensure that `spack env activate`, `spack env deactivate`, and `despacktivate` continue to work. - [x] Also ensure that activate and deactivate both work with `set -u`
2019-07-17feature: Allow developers to use Spack for partial builds (#12006)Greg Becker1-1/+1
Added new diy option.
2019-07-15bugfix: fix module() function in setup-env.sh (#12030)Todd Gamblin1-6/+9
Modify `setup-env.sh` to export fewer variables but to still set the module() function correctly.
2019-07-15Fix typo in module template (#12028)Adam J. Stewart1-1/+1
2019-07-12Copy --{use,no}-cache from install to bootstrap (#11786)Oliver Breitwieser1-1/+2
2019-07-12Add template creation test, --skip-editor option (#11970)Tamara Dahlgren1-1/+1
* Add template creation test * Added --skip-editor option to "spack create": normally "spack create" opens an editor for the user after generating a package file; when the --skip-editor option is used, "spack create" only generates the package file and does not open an editor * Added --skip-editor option to bash completion
2019-07-12bugfix: setup-env.sh detects bash even when bash is run as sh (#11994)Todd Gamblin1-7/+2
- `setup-env.sh` was not properly detecting a bash shell when bash was run as /bin/sh. - Detection routine now always reports bash when bash is run as sh, and no longer parses the path to the executable indicated in `$BASH`.
2019-07-05setup-env.sh: make setup scripts work with `set -u`Todd Gamblin2-4/+11
- Add set -u to the setup-env.sh test script - Refactor lines in setup-env.sh that tested potentially undefined variables to use the `[ -z ${var+x} ]` construct
2019-07-05tests: add tests for setup-env.shTodd Gamblin3-6/+331
- tests use a shell-script harness and test all Spack commands that require special shell support. - tests work in bash, zsh, and dash - run setup-env.sh tests on macos and linux builds. - we run them on macos and linux
2019-07-05setup-env.sh: now compatible with bash, zsh, and dashTodd Gamblin1-65/+114
- replace use of [[ with [ - replace function foo { .. } with foo() { .. } - wrap some long lines - add lsof and /proc/fd magic so that we can find the sourced file even in dash - only do the complicated shell checks in one place; test $_sp_shell elsewhere.
2019-06-29setup-env.sh: send cd output to /dev/null when it affects SpackTodd Gamblin1-9/+12
- We've seen this a few times now where users have set up `cd` to echo the new directory, and it screws up `setup-env.sh` - In the past we've said this is user error. - Here, we just fix it by sending `cd` output to /dev/null where needed. - this works in bash, zsh, and dash
2019-06-10CD: Package Index (#11665)Axel Huebl5-0/+53
Build the docker image for the new package index REST API on each merge/push to develop and master.
2019-06-08tests: restore `spack spec mpileaks` in test output for Python 2 (#11664)Todd Gamblin1-5/+3
- also remove coverage calls of these commands - we JUST want coverage to tell us what is in the test suite.
2019-06-05travis: disable `spack spec` for python 2.7 as well as 2.6Todd Gamblin1-1/+1
- keep this disabled until the `compiler find` fork bomb is fixed.
2019-05-27commands: completion for new --update and --header options (#11566)Todd Gamblin1-2/+2
2019-05-16junit: escape remaining inputs. (#11382)Matthias Wolf1-2/+2
2019-05-10bugfix: Do not run spack spec in the test environment for Python 2.6Todd Gamblin1-1/+3
- Trying to get our Python 2.6 tests working again. - This seems to crash travis in the 2.6 environment, for whatever reason.
2019-05-11coverage: only run coverage for subset of unit testsTodd Gamblin1-4/+5
- Codecov cannot handle as many coverage reports as we are generating - as a result, our PR coverage pages have been broken for a while, and it's hard to tell people where to enhance their testing in PR reviews. - Scale back to only running coverage for 3.7 and 2.7 unit tests - This is *probably* better. We run the build tests for good measure, but we do not need to evaluate them for coverage. The coverage reports are about unit tests.
2019-05-04Added a function that concretizes specs together (#11158)Massimiliano Culpo1-0/+15
* Added a function that concretizes specs together * Specs concretized together are copied instead of being referenced This makes the specs different objects and removes any reference to the fake root package that is needed currently for concretization. * Factored creating a repository for concretization into its own function * Added a test on overlapping dependencies
2019-05-02install spack under /opt/spack in docker images (#10969)Omar Padron1-18/+34
* install spack under /opt/spack
2019-04-24Fix handling of MODULEPATH in setup-env.sh (#11274)Toyohisa Kameyama1-1/+1
Usage of double quotes was preventing word-splitting when parsing module roots in setup-env.sh, which lead to an error when multiple module roots are used (in particular when Spack is pointed to use an upstream module root in addition to its own).
2019-04-16setup-env.sh: fix zsh compatibility error (#11153)paulanda1-3/+3
Still look for BASH_SOURCE[0] first, but if it's not set, _sp_source_file is initialized to an empty value addressing the unset parameter error (line 217).
2019-03-27spack chain (#8772)Peter Scheibel2-4/+20
* initial work to make use of an 'upstream' spack installation: this uses the DB of the upstream installation to check if a package is installed * need to query upstream dbs when adding new record to local db * prevent reindexing upstream DBs * set prefix on specs read from DB based on path stored in install record * check that Spack does not install packages that are recorded as installed in an upstream db * externals do not add their path to install records - need to use 'external_path' to get path of upstream externals * views need to check for upstream installations when linking metadata * package and spec now calculate upstream installation properties on-demand themselves rather than depending on concretization to set these properties up-front. The added tests for upstream installations don't work with this new strategy so they need to be updated * only refresh modules for local specs (not those in upstream packages); optionally generate local module files for packages installed upstream * when a user tries to locate a module file for a package installed upstream, tell them to use the upstream spack instance to locate it * support recursive upstream databases (allow upstream databases to use their own upstream databases) * separate upstream config into separate file with its own schema; each entry now also includes a name * metadata_dir is no longer customizable on a per-instance basis for YamlDirectoryLayout * treat metadata_dir as an instance variable but dont set it from kwargs; this follows several other hardcoded variables which must be consistent between upstream and downstream DBs. Also update DirectoryLayout.metadata_path to work entirely with Spec.prefix, since Spec.prefix is set from the DB when available (so metadata_path was duplicating that logic)
2019-03-25explicitly install unzip in all docker images (#10891)Omar Padron1-7/+10
2019-02-28Fix shell integration with environment-modules@4 (#10736)Michael Kuhn1-4/+10
2019-02-21release workflow: Add spack command for generating the .gitlab-ci.ymlScott Wittenburg1-0/+11
This spack command adds a new schema for a file which describes the builder containers available, along with the compilers availabe on each builder. The release-jobs command then generates the .gitlab-ci.yml file by first expanding the release spec set, concretizing each spec (in an appropriate docker container if --this-machine-only argument is not provided on command line), and then combining and staging all the concrete specs as jobs to be run by gitlab.
2019-02-21docker: Add Dockerfiles for images needed by AWS gitlab-runnersScott Wittenburg3-0/+118
The built images are set up with fairly recent versions of gcc and clang: - centos_7: [ gcc@5.5.0 (built from src), clang@6.0.0 (spack-built from src) ] - ubuntu_18.04: [ gcc@5.5.0 (system), clang@6.0.0-1ubuntu2 (system) ]
2019-02-21buildcache: Add sub-commands needed by release workflowScott Wittenburg1-0/+11
Adds four new sub-commands to the buildcache command: 1. save-yaml: Takes a root spec and a list of dependent spec names, along with a directory in which to save yaml files, and writes out the full spec.yaml for each of the dependent specs. This only needs to concretize the root spec once, then indexes it with the names of the dependent specs. 2. check: Checks a spec (via either an abstract spec or via a full spec.yaml) against remote mirror to see if it needs to be rebuilt. Comparies full_hash stored on remote mirror with full_hash computed locally to determine whether spec needs to be rebuilt. Can also generate list of specs to check against remote mirror by expanding the set of release specs expressed in etc/spack/defaults/release.yaml. 3. get-buildcache-name: Makes it possible to attempt to read directly the spec.yaml file on a remote or local mirror by providing the path where the file should live based on concretizing the spec. 4. download: Downloads all buildcache files associated with a spec on a remote mirror, including any .spack, .spec, and .cdashid files that might exist. Puts the files into the local path provided on the command line, and organizes them in the same hierarchy found on the remote mirror This commit also refactors lib/spack/spack/util/web.py to expose functionality allowing other modules to read data from a url.
2019-02-16[Docker] more base images (#10560)Omar Padron13-21/+90
* Revert "work around an issue with docker/dockerhub (#10547)" This reverts commit 0ef139ac4d475fb99e08b0856dd7f27b97082e50. * Add more docker base image configurations. - Renamed configs - "centos" -> "centos-7" - "fedora" -> "fedora-24" - "opensuse" -> "opensuse-tumbleweed" - "scilinux" -> "scilinux-7" - "ubuntu" -> "ubuntu-1804" - Added configs - "centos-6" - "fedora-29" - "opensuse-leap" - "scilinux-6" - "ubuntu-1604" With these changes, our docker hub image collection will consist of the following: - "spack/archlinux" - "latest" - "spack/centos" - "latest", "centos7", "7" - "centos6", "6" - "spack/fedora" - "latest", "29" - "24" - "spack/opensuse" - "latest", "tumbleweed" - "leap" - "spack/scilinux" - "latest", "7" - "6" - "spack/ubuntu" - "latest", "bionic", "18.04" - "xenial", "16.04"
2019-02-13make SPACK_SHELL detection more robust (#9712)Phil Carns1-1/+12
Spack shell detection in setup-env.sh was originally based on examining the executable name of $$ (from "ps"). In some cases this does not actually give the name of the shell used, for example when setup-env.sh was invoked from a script using "#!". To make shell detection more robust, this adds a preliminary check for shell variables which indicate that the shell is bash or zsh; the executable name of $$ is retained as a fallback if those variables are not defined.
2019-02-08work around an issue with docker/dockerhub (#10547)Omar Padron1-0/+14
* try to workaround an issue with docker/dockerhub * apply workaround only when running in travis
2019-01-24fix issue where docker images were being pushed without login (#10379)Omar Padron1-3/+1
2019-01-12fix typo in docker test script (#10334)Omar Padron1-1/+1
2019-01-12fix archlinux docker build (#10289)Omar Padron2-8/+7
2019-01-10spack versions: only list safe versions (#10004)Adam J. Stewart1-1/+1
* spack versions: only list safe versions * Add unit tests for spack versions -s