summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-10-28spack setup-env.sh: make zsh loading async compatible, and ~10x faster (in ↵Tom Scogland4-10/+26
some cases) (#26120) Currently spack is a bit of a bad actor as a zsh plugin, and it was my fault. The autoload and compinit should really be handled by the user, as was made abundantly clear when I found spack was doing completion initialization for *all* of my plugins due to a deferred setup that was getting messed up by it. Making this conditional took spack load time from 1.5 seconds (with module loading disabled) to 0.029 seconds. I can actually afford to load spack by default with this change in. Hopefully someday we'll do proper zsh completion support, but for now this helps a lot. * use zsh hist expansion in place of dirname * only run (bash)compinit if compdef/complete missing * add zsh compiled files to .gitignore * move changes to .in file, because spack
2021-10-28New version: py-fasteners 0.16.3 (#27006)iarspider1-1/+3
2021-10-28Remove failing macOS test (#27009)Harmen Stoppels1-14/+0
2021-10-28Add Pybind11 v2.8 (#26867)Brent Huisman1-1/+4
* Add Pybind11 v2.8 * Add Python dependency * Update package.py * Added Pybind v2.8.1
2021-10-28YamlFilesystemView: improve file removal performance via batching (#24355)Robert Blackwell3-17/+23
* Drastically improve YamlFilesystemView file removal via batching The `remove_file` routine has to check if the file is owned by multiple packages, so it doesn't remove necessary files. This is done by the `get_all_specs` routine, which walks the entire package tree. With large numbers of packages on shared file systems, this can take seconds per file tree traversal, which adds up extremely quickly. For example, a single deactivate of a largish python package in our software stack on GPFS took approximately 40 minutes. This patch simply replaces `remove_file` with a batch `remove_files` routine. This routine removes a list of files rather than a single file, requiring only one traversal per batch. In practice this means a package can be removed in seconds time, rather than potentially hours, essentially a ~100x speedup (ignoring initial deactivation logic, which takes about 3 minutes in our test setup).
2021-10-28cmake: add v3.21.4, v3.20.6 (#27004)Harmen Stoppels1-0/+2
2021-10-28dtcmp & lwgrp: add shared variant (#26999)Cameron Stanavige2-5/+20
2021-10-28Fix sbang hook for non-writable files (#27007)Michael Kuhn2-0/+15
* Fix sbang hook for non-writable files PR #26793 seems to have broken the sbang hook for files with missing write permissions. Installing perl now breaks with the following error: ``` ==> [2021-10-28-12:09:26.832759] Error: PermissionError: [Errno 13] Permission denied: '$SPACK/opt/spack/linux-fedora34-zen2/gcc-11.2.1/perl-5.34.0-afuweplnhphcojcowsc2mb5ngncmczk4/bin/cpanm' ``` Temporarily add write permissions to the original file so it can be overwritten with the patched one. And test that file permissions are preserved in sbang even for non-writable files Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-10-28nsimd: add v3.0.1, determine SIMD variant automatically by default (#26850)Erik Schnetter1-3/+35
2021-10-28buildcaches: fix directory link relocation (#26948)Paul Ferrell3-3/+62
When relocating a binary distribution, Spack only checks files to see if they are a link that needs to be relocated. Directories can be such links as well, however, and need to undergo the same checks and potential relocation.
2021-10-27llvm: use cmake helper functions (#26988)Seth R. Johnson1-101/+64
* llvm: use cmake helper functipack stns * llvm: review feedback
2021-10-27New versions: py-cffi 1.15.0, 1.14.6 (#26979)iarspider1-1/+2
* New versions: py-cffi 1.15.0, 1.14.6 * Changes from review
2021-10-27New versions of py-cachetools (#26976)iarspider1-0/+4
* New versions of py-cachetools * Changes from review
2021-10-27New version: py-bottle@0.12.19 (#26973)iarspider1-1/+2
* New version: py-bottle@0.12.19 * Changes from review
2021-10-27hpcviewer: add support for macosx, add version 2021.10 (#26823)Mark W. Krentel1-18/+78
2021-10-27Add w3emc version 2.9.1 (#26880)Kyle Gerheiser1-0/+1
2021-10-27New version: py-contextlib2 21.6.0 (#26985)iarspider1-3/+5
2021-10-27Remove documentation tests from GitHub Actions (#26981)Massimiliano Culpo3-61/+6
We moved documentation tests to readthedocs since a while, so remove the one on GitHub.
2021-10-27New versions: py-boken@2.3.3, 2.4.0, 2.4.1 (#26972)iarspider1-4/+23
2021-10-27New version: py-certifi 2021.10.8 (#26978)iarspider1-0/+1
2021-10-27hdf5-vol-log: add new package (#26956)H. Joe Lee1-0/+32
2021-10-27New version: py-commonmark 0.9.1 (#26983)iarspider1-1/+3
2021-10-27fftw: add v3.3.10 (#26982)Mosè Giordano1-0/+1
2021-10-27fontconfig: add v2.13.94 and fix test with dash (#26961)Bernhard Kaindl1-0/+5
Fix install --test=root with /bin/sh -> dash: A test uses SIGINT SIGTERM SIGABRT EXIT for trap -> use signal numbers
2021-10-27perl-extutils-installpaths: depend on perl-extutils-config (#26969)Bernhard Kaindl1-0/+2
2021-10-27strumpack: add v6.1.0, remove unused variants (#26971)Pieter Ghysels1-7/+2
2021-10-27recola: fix compilation (#26634)Valentin Volkl2-2/+5
* recola: fix compilation * Update var/spack/repos/builtin/packages/recola-sm/package.py Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov> * flake8 * fixes * fix typo * fix typo Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov>
2021-10-27tests: speed up `spack list` tests (#26958)Todd Gamblin2-36/+41
`spack list` tests are not using mock packages for some reason, and many are marked as potentially slow. This isn't really necessary; we don't need 6,000 packages to test the command. - [x] update tests to use `mock_packages` fixture - [x] remove `maybeslow` annotations
2021-10-27Allow non-UTF-8 encoding in sbang hook (#26793)Harmen Stoppels2-71/+181
Currently Spack reads full files containing shebangs to memory as strings, meaning Spack would have to guess their encoding. Currently Spack has a fixed guess of UTF-8. This is unnecessary, since e.g. the Linux kernel does not assume an encoding on paths at all, it's just bytes and some delimiters on the byte level. This commit does the following: 1. Shebangs are treated as bytes, so that e.g. latin1 encoded files do not throw UnicodeEncoding errors, and adds a test for this. 2. No more bytes than necessary are read to memory, we only have to read until the first newline, and from there on we an copy the file byte by bytes instead of decoding and re-encoding text. 3. We cap the number of bytes read to 4096, if no newline is found before that, we don't attempt to patch it. 4. Add support for luajit too. This should make Spack both more efficient and usable for non-UTF8 files.
2021-10-27Fix assumption v.concrete => isinstance(v, Version) (#26537)Harmen Stoppels2-2/+23
* Add test * Only extend with Git version when using Version * xfail v.concrete test
2021-10-26config: overrides for caches and system and user scopes (#26735)Harmen Stoppels9-53/+234
Spack's `system` and `user` scopes provide ways for administrators and users to set global defaults for all Spack instances, but for use cases where one wants a clean Spack installation, these scopes can be undesirable. For example, users may want to opt out of global system configuration, or they may want to ignore their own home directory settings when running in a continuous integration environment. Spack also, by default, keeps various caches and user data in `~/.spack`, but users may want to override these locations. Spack provides three environment variables that allow you to override or opt out of configuration locations: * `SPACK_USER_CONFIG_PATH`: Override the path to use for the `user` (`~/.spack`) scope. * `SPACK_SYSTEM_CONFIG_PATH`: Override the path to use for the `system` (`/etc/spack`) scope. * `SPACK_DISABLE_LOCAL_CONFIG`: set this environment variable to completely disable *both* the system and user configuration directories. Spack will only consider its own defaults and `site` configuration locations. And one that allows you to move the default cache location: * `SPACK_USER_CACHE_PATH`: Override the default path to use for user data (misc_cache, tests, reports, etc.) With these settings, if you want to isolate Spack in a CI environment, you can do this: export SPACK_DISABLE_LOCAL_CONFIG=true export SPACK_USER_CACHE_PATH=/tmp/spack This is a stop-gap approach until we have figured out how to deal with the system and user config scopes more generally, as there are plans to potentially / eventually get rid of them. **User config** Spack is a bit of a pain when you have: - a shared $HOME folder across different systems. - multiple Spack versions on the same system. **System config** - On shared systems with a versioned programming environment / toolkit, system administrators want to provide config for each version (e.g. 21.09, 21.10) of the programming environment, and the user Spack instance should be able to pick this up without a steep learning curve. - On shared systems the user should be able to opt out of the hard-coded config scope in /etc/spack, since it may be incompatible with their particular instance. Currently Spack can only opt out of all config scopes through overrides with `"config:":`, `"packages:":`, but that also drops the defaults config, which would have to be repeated, which is undesirable, especially the lengthy packages.yaml. An example use case is: having config in this folder: ``` /path/to/programming/environment/{version}/{compilers,packages}.yaml ``` and have `module load spack-system-config` set the variable ``` SPACK_SYSTEM_CONFIG_PATH=/path/to/programming/environment/{version} ``` where the user no longer has to worry about what `{version}` they are on. **Continuous integration** Finally, there is the use case of continuous integration, which may clone an arbitrary Spack version, which optimally should not pick up system or user config from the previous run (like may happen in classical bare metal non-containerized filesystem side effect ridden jenkins pipelines). In fact this is very similar to how spack itself tries to avoid picking up system dependencies during builds... **But environments solve this?** - You could do `include`s in environment files to get similar behavior to the spack_system_config_path example, but environments require you to: 1) require paths to individual config files, not directories. 2) fail if the listed config file does not exist - They allow you to override config scopes, but this is generally too rigurous, as it requires you to repeat the default config, in particular packages.yaml, and just defies the point of layered config. Co-authored-by: Tom Scogland <tscogland@llnl.gov> Co-authored-by: Tim Fuller <tjfulle@sandia.gov> Co-authored-by: Steve Leak <sleak@lbl.gov> Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2021-10-26Build OpenMP in LLVM via LLVM_ENABLE_RUNTIMES. (#26870)Ye Luo1-5/+34
2021-10-26fix bug when there is version id in the path of compiler (#26916)Hao Lyu1-2/+2
2021-10-26modules: allow user to remove arch dir (#24156)Greg Becker8-11/+58
* allow no arch-dir modules * add tests for modules with no arch * document arch-specific module roots
2021-10-26hpctoolkit: add version 2021.10.15 (#26881)Mark W. Krentel1-0/+1
2021-10-26Py-Libensemble: Add E4S testsuite stand alone test (#26270)Richarda Butler1-0/+26
2021-10-26fix the spack test dir (#26816)Miroslav Stoyanov1-1/+1
2021-10-26ROCm packages: add RelWithDebInfo build_type (#26888)Sreenivasa Murthy Kolam41-24/+57
Also set default build_type to Release for many ROCm packages.
2021-10-26modules: configurable module defaults (#24367)Greg Becker4-3/+86
Any spec satisfying a default will be symlinked to `default` If multiple specs have modulefiles in the same directory and satisfy configured module defaults, then whichever was written last will be default.
2021-10-26upp: Add version 10.0.10 (#26946)Kyle Gerheiser1-0/+2
2021-10-26htop: add new URL and versions (#26928)Seth R. Johnson1-6/+12
2021-10-26virtest: prevent out-of-order build/test (#26944)Ben Morgan1-0/+5
Use of `-R` flag to CTest command causes "empty-14" test to run, by matching "empty", before the empty-14 target is built. Patch CTest command in buildscript to match name exactly.
2021-10-26hiop: update constraints and add new version (#26905)Asher Mancinelli1-50/+51
* Update hiop package dependencies * Use single quotes to shrink diff * add hiop 0.5.1 * apply flake8 * Apply formatting suggestions
2021-10-26Add checksum for py-argon2-cffi 21.1.0 and update python dependency (#26894)iarspider1-1/+4
* Add checksum for py-argon2-cffi 21.1.0 and update python dependency * Update package.py
2021-10-26New versions: py-bleach 4.0.0 and 4.1.0 (#26947)iarspider1-0/+3
2021-10-26New version: py-beniget 0.4.1 (#26945)iarspider1-0/+1
2021-10-26geant4: depends_on vecgeom@1.1.8:1.1 range (#26917)Wouter Deconinck2-1/+2
* [geant4] depends_on vecgeom@1.1.8:1.1 range While previous versions were unclear, the [geant4.10.7 release notes](https://geant4-data.web.cern.ch/ReleaseNotes/ReleaseNotes4.10.7.html) indicate that vecgeom@1.1.8 is a minimum required version, not an exact required version ("Set VecGeom-1.1.8 as minimum required version for optional build with VecGeom."). This will allow some more freedom on the concretizer solutions while allowing geant4 to take advantage of bugfixes and improvements in vecgeom. * [vecgeom] new version 1.1.17
2021-10-26freetype: explicitly specify dependencies (#26942)Seth R. Johnson1-2/+8
Freetype picked up 'brotli' from homebrew, causing issues downstream.
2021-10-26tau: add version 2.30.2 (#26941)eugeneswalker1-0/+1
2021-10-26unifyfs: Remove the hdf5 variant (examples only) (#26932)Chuck Atkins2-12/+1
UnifyFS doesn't actually use HDF5 for anything. Enabling it only enables a few examples to be built so it's not actually a dependency of the package.