summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-10-26config: overrides for caches and system and user scopes (#26735)Harmen Stoppels9-53/+234
Spack's `system` and `user` scopes provide ways for administrators and users to set global defaults for all Spack instances, but for use cases where one wants a clean Spack installation, these scopes can be undesirable. For example, users may want to opt out of global system configuration, or they may want to ignore their own home directory settings when running in a continuous integration environment. Spack also, by default, keeps various caches and user data in `~/.spack`, but users may want to override these locations. Spack provides three environment variables that allow you to override or opt out of configuration locations: * `SPACK_USER_CONFIG_PATH`: Override the path to use for the `user` (`~/.spack`) scope. * `SPACK_SYSTEM_CONFIG_PATH`: Override the path to use for the `system` (`/etc/spack`) scope. * `SPACK_DISABLE_LOCAL_CONFIG`: set this environment variable to completely disable *both* the system and user configuration directories. Spack will only consider its own defaults and `site` configuration locations. And one that allows you to move the default cache location: * `SPACK_USER_CACHE_PATH`: Override the default path to use for user data (misc_cache, tests, reports, etc.) With these settings, if you want to isolate Spack in a CI environment, you can do this: export SPACK_DISABLE_LOCAL_CONFIG=true export SPACK_USER_CACHE_PATH=/tmp/spack This is a stop-gap approach until we have figured out how to deal with the system and user config scopes more generally, as there are plans to potentially / eventually get rid of them. **User config** Spack is a bit of a pain when you have: - a shared $HOME folder across different systems. - multiple Spack versions on the same system. **System config** - On shared systems with a versioned programming environment / toolkit, system administrators want to provide config for each version (e.g. 21.09, 21.10) of the programming environment, and the user Spack instance should be able to pick this up without a steep learning curve. - On shared systems the user should be able to opt out of the hard-coded config scope in /etc/spack, since it may be incompatible with their particular instance. Currently Spack can only opt out of all config scopes through overrides with `"config:":`, `"packages:":`, but that also drops the defaults config, which would have to be repeated, which is undesirable, especially the lengthy packages.yaml. An example use case is: having config in this folder: ``` /path/to/programming/environment/{version}/{compilers,packages}.yaml ``` and have `module load spack-system-config` set the variable ``` SPACK_SYSTEM_CONFIG_PATH=/path/to/programming/environment/{version} ``` where the user no longer has to worry about what `{version}` they are on. **Continuous integration** Finally, there is the use case of continuous integration, which may clone an arbitrary Spack version, which optimally should not pick up system or user config from the previous run (like may happen in classical bare metal non-containerized filesystem side effect ridden jenkins pipelines). In fact this is very similar to how spack itself tries to avoid picking up system dependencies during builds... **But environments solve this?** - You could do `include`s in environment files to get similar behavior to the spack_system_config_path example, but environments require you to: 1) require paths to individual config files, not directories. 2) fail if the listed config file does not exist - They allow you to override config scopes, but this is generally too rigurous, as it requires you to repeat the default config, in particular packages.yaml, and just defies the point of layered config. Co-authored-by: Tom Scogland <tscogland@llnl.gov> Co-authored-by: Tim Fuller <tjfulle@sandia.gov> Co-authored-by: Steve Leak <sleak@lbl.gov> Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2021-10-26Build OpenMP in LLVM via LLVM_ENABLE_RUNTIMES. (#26870)Ye Luo1-5/+34
2021-10-26fix bug when there is version id in the path of compiler (#26916)Hao Lyu1-2/+2
2021-10-26modules: allow user to remove arch dir (#24156)Greg Becker8-11/+58
* allow no arch-dir modules * add tests for modules with no arch * document arch-specific module roots
2021-10-26hpctoolkit: add version 2021.10.15 (#26881)Mark W. Krentel1-0/+1
2021-10-26Py-Libensemble: Add E4S testsuite stand alone test (#26270)Richarda Butler1-0/+26
2021-10-26fix the spack test dir (#26816)Miroslav Stoyanov1-1/+1
2021-10-26ROCm packages: add RelWithDebInfo build_type (#26888)Sreenivasa Murthy Kolam41-24/+57
Also set default build_type to Release for many ROCm packages.
2021-10-26modules: configurable module defaults (#24367)Greg Becker4-3/+86
Any spec satisfying a default will be symlinked to `default` If multiple specs have modulefiles in the same directory and satisfy configured module defaults, then whichever was written last will be default.
2021-10-26upp: Add version 10.0.10 (#26946)Kyle Gerheiser1-0/+2
2021-10-26htop: add new URL and versions (#26928)Seth R. Johnson1-6/+12
2021-10-26virtest: prevent out-of-order build/test (#26944)Ben Morgan1-0/+5
Use of `-R` flag to CTest command causes "empty-14" test to run, by matching "empty", before the empty-14 target is built. Patch CTest command in buildscript to match name exactly.
2021-10-26hiop: update constraints and add new version (#26905)Asher Mancinelli1-50/+51
* Update hiop package dependencies * Use single quotes to shrink diff * add hiop 0.5.1 * apply flake8 * Apply formatting suggestions
2021-10-26Add checksum for py-argon2-cffi 21.1.0 and update python dependency (#26894)iarspider1-1/+4
* Add checksum for py-argon2-cffi 21.1.0 and update python dependency * Update package.py
2021-10-26New versions: py-bleach 4.0.0 and 4.1.0 (#26947)iarspider1-0/+3
2021-10-26New version: py-beniget 0.4.1 (#26945)iarspider1-0/+1
2021-10-26geant4: depends_on vecgeom@1.1.8:1.1 range (#26917)Wouter Deconinck2-1/+2
* [geant4] depends_on vecgeom@1.1.8:1.1 range While previous versions were unclear, the [geant4.10.7 release notes](https://geant4-data.web.cern.ch/ReleaseNotes/ReleaseNotes4.10.7.html) indicate that vecgeom@1.1.8 is a minimum required version, not an exact required version ("Set VecGeom-1.1.8 as minimum required version for optional build with VecGeom."). This will allow some more freedom on the concretizer solutions while allowing geant4 to take advantage of bugfixes and improvements in vecgeom. * [vecgeom] new version 1.1.17
2021-10-26freetype: explicitly specify dependencies (#26942)Seth R. Johnson1-2/+8
Freetype picked up 'brotli' from homebrew, causing issues downstream.
2021-10-26tau: add version 2.30.2 (#26941)eugeneswalker1-0/+1
2021-10-26unifyfs: Remove the hdf5 variant (examples only) (#26932)Chuck Atkins2-12/+1
UnifyFS doesn't actually use HDF5 for anything. Enabling it only enables a few examples to be built so it's not actually a dependency of the package.
2021-10-25qt package: versions @:5.13 don't build with gcc@11: (#26922)Seth R. Johnson1-0/+1
2021-10-25charmpp: add version 7.0.0 (#26940)Ronak Buch1-0/+1
2021-10-25py-kornia: add version 0.6.1 (#26939)Adam J. Stewart1-0/+3
2021-10-25py-shapely: add version 1.8.0 (#26937)Adam J. Stewart1-2/+4
2021-10-25py-scikit-learn: add version 1.0.1 (#26934)Adam J. Stewart1-0/+1
2021-10-25PyTorch/torchvision: add version 1.10.0/0.11.1 (#26889)Adam J. Stewart2-21/+16
* For py-torch: Also update dependencies: many version constraints with an upper bound of @1.9 are now open (e.g. `@1.8.0:1.9` is converted to `@1.8.0:`). * For py-torchvision: Also add 0.11.0 and update ^pil constraint to avoid building with 8.3.0
2021-10-25Add new versions of py-autopep8 and py-pycodestyle (#26924)iarspider2-15/+35
* Add new versions of py-autopep8 (1.5.7, 1.6.0) and py-pycodestyle (2.7.0, 2.8.0) * Update package.py * Restore old versions
2021-10-25deal.II package: Update CMake variable for >=9.3.0 (#26909)Daniel Arndt1-3/+8
2021-10-25containerize: pin the Spack version used in a container (#21910)Massimiliano Culpo22-90/+628
This PR permits to specify the `url` and `ref` of the Spack instance used in a container recipe simply by expanding the YAML schema as outlined in #20442: ```yaml container: images: os: amazonlinux:2 spack: ref: develop resolve_sha: true ``` The `resolve_sha` option, if true, verifies the `ref` by cloning the Spack repository in a temporary directory and transforming any tag or branch name to a commit sha. When this new ability is leveraged an additional "bootstrap" stage is added, which builds an image with Spack setup and ready to install software. The Spack repository to be used can be customized with the `url` keyword under `spack`. Modifications: - [x] Permit to pin the version of Spack, either by branch or tag or sha - [x] Added a few new OSes (centos:8, amazonlinux:2, ubuntu:20.04, alpine:3, cuda:11.2.1) - [x] Permit to print the bootstrap image as a standalone - [x] Add documentation on the new part of the schema - [x] Add unit tests for different use cases
2021-10-25py-avro: new version 1.10.2 (#26927)iarspider1-0/+2
* Add checksum for py-avro@1.10.2 * Update package.py
2021-10-25cuda: add 11.4.1, 11.4.2, 11.5.0. (#26892)Olli Lupton3-4/+21
* cuda: add 11.4.1, 11.4.2, 11.5.0. Note that the curses dependency from cuda-gdb was dropped in 11.4.0. * Update clang/gcc constraints. * Address review, assume clang 12 is OK from 11.4.1 onwards. * superlu-dist@7.1.0 conflicts with cuda@11.5.0. * Update var/spack/repos/builtin/packages/superlu-dist/package.py Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com> Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-10-25Add checksum for py-atomicwrites@1.4.0 (#26923)iarspider1-0/+1
2021-10-25draco: new version 7_12_0 (#26907)Kendra Long!1-0/+1
* Add draco-7_12_0 to package file * Update hash to zip version
2021-10-25eospac: new default version 6.5.0 (#26723)Kelly (KT) Thompson1-23/+58
* [pkg][new version] Provide eospac@6.5.0 and mark it as default. * Merge in changes found in #21629 * Mark all alpha/beta versions as deprecated. - Addresses @sethrj's recommendation - Also add a note indicated why these versions are marked this way.
2021-10-25Reduce verbosity of module files warningHarmen Stoppels1-3/+1
1. Currently it prints not just the spec name, but the dependencies + their variants + their compilers + their architectures + ... 2. It's clear from the context what spec the message applies to, so, let's not print the spec at all.
2021-10-25silo: new release 4.11 (#26876)Olivier Cessenat1-2/+17
2021-10-25py-nipype: add 1.7.0 (#26883)Manuela Kuhn1-1/+2
2021-10-25Return early in do_fetch when there is no code or a package is external (#26926)Harmen Stoppels3-7/+32
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2021-10-25virtuals: simplify virtual handlingTodd Gamblin1-16/+5
These three rules in `concretize.lp` are overly complex: ```prolog :- not provider(Package, Virtual), provides_virtual(Package, Virtual), virtual_node(Virtual). ``` ```prolog :- provides_virtual(Package, V1), provides_virtual(Package, V2), V1 != V2, provider(Package, V1), not provider(Package, V2), virtual_node(V1), virtual_node(V2). ``` ```prolog provider(Package, Virtual) :- root(Package), provides_virtual(Package, Virtual). ``` and they can be simplified to just: ```prolog provider(Package, Virtual) :- node(Package), provides_virtual(Package, Virtual). ``` - [x] simplify virtual rules to just one implication - [x] rename `provides_virtual` to `virtual_condition_holds`
2021-10-25Add a unit test to prevent regressionMassimiliano Culpo6-2/+56
2021-10-25ASP-based solver: a package eligible to provide a virtual must provide itMassimiliano Culpo1-0/+5
fixes #26866 This semantics fits with the way Spack currently treats providers of virtual dependencies. It needs to be revisited when #15569 is reworked with a new syntax.
2021-10-25berkeleygw: force openmp propagation on some providers of blas / ffw-api ↵Miguel Dias Costa1-7/+8
(#26918)
2021-10-24hyperfine: new package (#26901)Michael Kuhn1-0/+23
2021-10-24Add new math solver versions (#26903)Satish Balay5-0/+7
* tasmanian: add @7.7 * butterflypack: add @2.0.0 * pflotran: add @3.0.2 * alquimia: add @1.0.9 * superlu-dist: add @7.1.1
2021-10-24meson: add 0.60.0 (#26921)Michael Kuhn1-0/+1
2021-10-24Mark flaky test_ci_rebuild as xfail (#26911)Scott Wittenburg1-0/+1
2021-10-24Fix pkg-config dependencies (#26912)Michael Kuhn9-9/+9
pkg-config and pkgconf are implementations of the pkgconfig provider.
2021-10-24glib: add 2.70.0 (#26915)Michael Kuhn2-1/+33
2021-10-24gcc: support alternate mechanism for providing GCC rpaths (#26590)Seth R. Johnson1-6/+59
* gcc: support runtime ability to not install spack rpaths Fixes #26582 . * gcc: Fix malformed specs file and add docs The updated docs point out that the spack-modified GCC does *not* follow the usual behavior of LD_RUN_PATH! * gcc: fix bad rpath on macOS This bug has been around since the beginning of the GCC package file: the rpath command it generates for macOS writes a single (invalid) rpath entry. * gcc: only write rpaths for directories with shared libraries The original lib64+lib was just a hack for "in case either has one" but it's easy to tell whether either has libraries that can be directly referenced.
2021-10-24python: add 3.9.7, 3.8.12, 3.7.12 and 3.6.15 (#26914)Michael Kuhn1-1/+5