summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-08-26New package: reprimand (#25364)Erik Schnetter2-0/+89
2021-08-26Conditionally remove 'context' from kwargs in _urlopen (#25316)Paul Kuberry1-3/+9
* Conditionally remove 'context' from kwargs in _urlopen Previously, 'context' is purged from kwargs in _urlopen to conform to varying support for 'context' in different versions of urllib. This fix tries to use 'context', and then removes it if an exception is thrown and tries again. * Specify error type in try statement in _urlopen Specify TypeError when checking if 'context' is in kwargs for _urlopen. Also, if try fails, check that 'context' is in the error message before removing from kwargs.
2021-08-26Speedup environment activation, part 2 (#25633)Adam J. Stewart5-113/+90
This is a direct followup to #13557 which caches additional attributes that were added in #24095 that are expensive to compute. I had to reopen #25556 in another PR to invalidate the GitLab CI cache, but see #25556 for prior discussion. ### Before ```console $ time spack env activate . real 2m13.037s user 1m25.584s sys 0m43.654s $ time spack env view regenerate ==> Updating view at /Users/Adam/.spack/.spack-env/view real 16m3.541s user 10m28.892s sys 4m57.816s $ time spack env deactivate real 2m30.974s user 1m38.090s sys 0m49.781s ``` ### After ```console $ time spack env activate . real 0m8.937s user 0m7.323s sys 0m1.074s $ time spack env view regenerate ==> Updating view at /Users/Adam/.spack/.spack-env/view real 2m22.024s user 1m44.739s sys 0m30.717s $ time spack env deactivate real 0m10.398s user 0m8.414s sys 0m1.630s ``` Fixes #25555 Fixes #25541 * Speedup environment activation, part 2 * Only query distutils a single time * Fix KeyError bug * Make vermin happy * Manual memoize * Add comment on cross-compiling * Use platform-specific include directory * Fix multiple bugs * Fix python_inc discrepancy * Fix import tests
2021-08-26aws-parallelcluster: add v2.11.2 (#25635)Enrico Usai1-1/+2
Signed-off-by: Rex <shuningc@amazon.com>
2021-08-26fixing "problems in speck" as identified by repology (#25491)Vanessasaurus378-393/+394
Most of these are perl packages that need to point to the meta docs site, and then a fair amount of http addresses that need to be https, and then the rest are usually documentation sites that no longer exist or were otherwise changes Signed-off-by: vsoch <vsoch@users.noreply.github.com> Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2021-08-26Set pubkey trust to ultimate during `gpg trust` (#24976)Harmen Stoppels1-3/+42
* Set pubkey trust to ultimate during `gpg trust` Tries to solve the same problem as #24760 without surpressing stderr from gpg commands. This PR makes every imported key trusted in the gpg database. Note: I've outlined [here](https://github.com/spack/spack/pull/24760#issuecomment-883183175) that gpg's trust model makes sense, since how can we trust a random public key we download from a binary cache? * Fix test
2021-08-26Ensure environment are deactivated when bootstrapping (#25607)Massimiliano Culpo5-23/+52
Fixes #25603 This commit adds a new context manager to temporarily deactivate active environments. This context manager is used when setting up bootstrapping configuration to make sure that the current environment is not affected by operations on the bootstrap store. * Preserve exit code 1 if nothing is found * Use context manager for the environment
2021-08-26opencl-c-headers and opencl-clhpp: add new versions (#25576)lorddavidiii2-0/+4
2021-08-26py-omegaconf: add missing dependency (#25589)Adam J. Stewart1-0/+1
2021-08-26opium package: add version 4.1 and update blas/lapack dependencies (#25591)Glenn Johnson1-12/+17
- remove unneeded dependency on blas - create external-lapack variant - patch makefile to not build lapack if `+external-lapack` Also: - fix homepage link - set parallel = False - make references to `spec` consistent - remove unneeded `build` method
2021-08-26Remove references to self.install_test_root from packaging guide (#25238)Tamara Dahlgren1-27/+19
2021-08-26Avoid double loop in subprocess_context.store_patches (#25621)Massimiliano Culpo1-10/+9
fixes #21643 As far as I can see the double loop is not needed, since "patch" is never used and the items in the list are tuples of three values.
2021-08-26Remove fork_context from llnl.util.lang (#25620)Massimiliano Culpo1-18/+0
This object was introduced in #18124, and was later superseded by #18205 and removed any use if the object.
2021-08-26Regression test for version selection with preferences (#25602)Massimiliano Culpo5-0/+60
This commit adds a regression test for version selection with preferences in `packages.yaml`. Before PR 25585 we used negative weights in a minimization to select the optimal version. This may lead to situations where a dependency may make the version score of dependents "better" if it is preferred in packages.yaml.
2021-08-26zstd package: use make instead of CMake (#25610)Harmen Stoppels2-32/+17
zstd doesn't use -pthread for the static lib anymore, so gcc should be fine
2021-08-26VTK: add version 9.0.3 (#25609)Rémi Lacroix1-0/+1
2021-08-26fixing bugs with new package updates for samtools and abi-dum (#25615)Vanessasaurus2-0/+3
samtools needed more constraints for htslib, and abi-dumper was missing pkg-config on the dependency universal ctags Signed-off-by: vsoch <vsoch@users.noreply.github.com> Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2021-08-25Bugfix: reinstalling updated develop specs (#25579)Harmen Stoppels1-6/+2
PackageInstaller and Package.installed disagree over what it means for a package to be installed: PackageInstaller believes it should be enough for a database entry to exist, whereas Package.installed requires a database entry & a prefix directory. This leads to the following niche issue: * a develop spec in an environment is successfully installed * then somehow its install prefix is removed (e.g. through a bug fixed in #25583) * you modify the sources and reinstall the environment 1. spack checks pkg.installed and realizes the develop spec is NOT installed, therefore it doesn't need to have 'overwrite: true' 2. the installer gets the build task and checks the database and realizes the spec IS installed, hence it doesn't have to install it. 3. the develop spec is not rebuilt. The solution is to make PackageInstaller and pkg.installed agree over what it means to be installed, and this PR does that by dropping the prefix directory check from pkg.installed, so that it only checks the database. As a result, spack will create a build task with overwrite: true for the develop spec, and the installer in fact handles overwrite requests fine even if the install prefix doesn't exist (it just does a normal install).
2021-08-26py-grpcio: set parallel build jobs (#25616)Xiao-Yong1-0/+3
By default the number of parellel compiler processes launched by py-grpcio equals the number of threads. This commit limit it to spack config build_jobs.
2021-08-25New Package: perl-fth (#21879)Olivier Cessenat3-0/+140
2021-08-25[pkg][new version] Provide eospac@6.5.0beta (#25614)Kelly (KT) Thompson1-2/+7
* Provide new version of eospac. + Provide version 6.5.0beta. + Make version 6.4.2 the default + Also increment * volunteer to be the maintainer (for now).
2021-08-25environment: match concrete specs only if they have the same build hash (#25575)Massimiliano Culpo1-0/+15
see #25563 When we have a concrete environment and we ask to install a concrete spec from a file, currently Spack returns a list of specs that are all the one that match the argument DAG hash. Instead we want to compare build hashes, which also account for build-only dependencies.
2021-08-25[py-meshio] Added py-meshio package (#25578)Matthieu Dorier2-0/+34
* Added py-meshio package * Added setuptools dependency to py-meshio package * Update var/spack/repos/builtin/packages/py-meshio/package.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-meshio/package.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-meshio/package.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-meshio/package.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-meshio/package.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * added missing py-importlib-metadata dependency in py-meshio Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2021-08-25installation: filter padding from binary installs, too (#25600)Todd Gamblin1-2/+7
#25303 filtered padding from build output, but it's still there in binary install/relocate output, so our CI logs are still quite long and frequently hit the limit. - [x] add context handler from #25303 to buildcache installation as well
2021-08-25Make `spack graph -i` environment-aware (#25599)Todd Gamblin2-2/+18
This allows you to run `spack graph --installed` from within an environment and get a dot graph of its concrete specs. - [x] make `spack graph -i` environment-aware - [x] add code to the generated dot graph to ensure roots have min rank (i.e., they're all at the top or left of the DAG)
2021-08-25Bump up version for rocm-debug-agent - ROCm 4.3.0 (#25588)arjun-raj-kuppala2-3/+18
2021-08-25libarchive: add v3.5.2, add maintainer, reworked recipe (#25604)Harmen Stoppels1-9/+40
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-08-25ca-certificates-mozilla: add v2021-07-05 (#25565)Harmen Stoppels1-0/+2
2021-08-25libuv: add v1.42.0, v1.41.1 (#25560)Harmen Stoppels1-0/+2
2021-08-24Update the cray-mpich package to use the new cray-mpich MPI wrappers (#25597)lukebroskop1-9/+23
As of cray-mpich version 8.1.7, conventional MPI compiler wrappers are included in cray-mpich. Co-authored-by: Luke Roskop <lroskop@cedar.head.cm.us.cray.com>
2021-08-24bootstrap: use `sys.exec_prefix` to set up external python correctly (#25593)Todd Gamblin1-1/+1
Bootstrapping clingo on macOS on `develop` gives errors like this: ``` ==> Error: RuntimeError: Unable to locate python command in /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/bin /Users/gamblin2/Workspace/spack/var/spack/repos/builtin/packages/python/package.py:662, in command: 659 return Executable(path) 660 else: 661 msg = 'Unable to locate {0} command in {1}' >> 662 raise RuntimeError(msg.format(self.name, self.prefix.bin)) ``` On macOS, `python` is laid out differently. In particular, `sys.executable` is here: ```console Python 2.7.16 (default, May 8 2021, 11:48:02) [GCC Apple LLVM 12.0.5 (clang-1205.0.19.59.6) [+internal-os, ptrauth-isa=deploy on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.executable '/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python' ``` Based on that, you'd think that `/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents` would be where you'd look for a `bin` directory, but you (and Spack) would be wrong: ```console $ ls /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/ Info.plist MacOS/ PkgInfo Resources/ _CodeSignature/ version.plist ``` You need to look in `sys.exec_prefix` ``` >>> sys.exec_prefix '/System/Library/Frameworks/Python.framework/Versions/2.7' ``` Which looks much more like a standard prefix, with understandable `bin`, `lib`, and `include` directories: ```console $ ls /System/Library/Frameworks/Python.framework/Versions/2.7 Extras/ Mac/ Resources/ bin/ lib/ Headers@ Python* _CodeSignature/ include/ $ ls -l /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python lrwxr-xr-x 1 root wheel 7B Jan 1 2020 /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python@ -> python2 ``` - [x] change `bootstrap.py` to use the `sys.exec_prefix` as the external prefix, instead of just getting the parent directory of the executable.
2021-08-25bugfix: Correct source of PID for -ddd installation outputs (#25596)Tamara Dahlgren1-2/+2
2021-08-24bootstrap: fix printing for python 2 (#25592)Todd Gamblin2-0/+4
2021-08-24locks: only open lockfiles once instead of for every lock held (#24794)Todd Gamblin1-20/+128
This adds lockfile tracking to Spack's lock mechanism, so that we ensure that there is only one open file descriptor per inode. The `fcntl` locks that Spack uses are associated with an inode and a process. This is convenient, because if a process exits, it releases its locks. Unfortunately, this also means that if you close a file, *all* locks associated with that file's inode are released, regardless of whether the process has any other open file descriptors on it. Because of this, we need to track open lock files so that we only close them when a process no longer needs them. We do this by tracking each lockfile by its inode and process id. This has several nice properties: 1. Tracking by pid ensures that, if we fork, we don't inadvertently track the parent process's lockfiles. `fcntl` locks are not inherited across forks, so we'll just track new lockfiles in the child. 2. Tracking by inode ensures that referencs are counted per inode, and that we don't inadvertently close a file whose inode still has open locks. 3. Tracking by both pid and inode ensures that we only open lockfiles the minimum number of times necessary for the locks we have. Note: as mentioned elsewhere, these locks aren't thread safe -- they're designed to work in Python and assume the GIL. Tasks: - [x] Introduce an `OpenFileTracker` class to track open file descriptors by inode. - [x] Reference-count open file descriptors and only close them if they're no longer needed (this avoids inadvertently releasing locks that should not be released).
2021-08-24openssl: new version 1.1.1l (#25586)Andrew W Elble1-1/+2
security update
2021-08-24Fix bindist network issues (#25587)Harmen Stoppels4-13/+10
* Fix bindist network issues * Another one using the network
2021-08-24m4: fixes for the NVIDIA HPC SDK (#25546)Scott McMillan3-0/+36
Co-authored-by: Scott McMillan <smcmillan@nvidia.com>
2021-08-24ASP-based solver: rework version facts (#25585)Massimiliano Culpo2-87/+105
This commit rework version facts so that: 1. All the information on versions is collected before emitting the facts 2. The same kind of atom is emitted for versions stemming from different origins (package.py vs. packages.yaml) In the end all the possible versions for a given package are totally ordered and they are given different and increasing weights staring from zero. This refactor allow us to avoid using negative weights, which in some configurations may make parent node score "better" and lead to unexpected "optimal" results.
2021-08-24Melissa: add v0.7.1, deprecate v0.7.0 (#25584)Christoph Conrads1-1/+2
2021-08-24petsc: added variants and dips (#24725)corentin-dev9-0/+225
Add HPDDM, MMG, ParMMG and Tetgen to PETSc. Add mmg version 5.5.2 (compatible with PETSc). Add parmmg, depending on mmg. Add pic variant to tetgen for PETSc.
2021-08-24fixing small bug that a line of spack monitor commands are still produced ↵Vanessasaurus1-1/+4
(#25366) Signed-off-by: vsoch <vsoch@users.noreply.github.com> Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2021-08-23Spelling fixes (#25570)Paul Spencer1-2/+2
2021-08-23Adding a heap of NOAA packages for UFS. (#25542)Timothy Brown20-0/+853
* Adding a heap of NOAA packages for UFS. Adding the Unified Forecast System (UFS) and all of the packages it depends on. * Fixing style tests. * Removing the package CMAKE_BUILD_TYPE override. * Removing compiler specs from `cmake_args()`.
2021-08-23re2c: add versions up to v2.2 (#25500)Harmen Stoppels2-2/+18
2021-08-23curl: add tls multi-valued variant, fix macOS build (#25553)Adam J. Stewart1-8/+71
2021-08-23py-numpy: add v1.21.2 (#25436)Adam J. Stewart1-5/+8
2021-08-23py-ipykernel: add v6.2.0 and v5.5.5 (#25520)Adam J. Stewart1-4/+14
2021-08-22py-pythran: add OpenMP dependency (#25137)Adam J. Stewart1-0/+32
2021-08-21Document how to handle changing build systems (#25174)Adam J. Stewart3-0/+352
2021-08-21ascent: a few small changes to the package (#25551)Cyrus Harrison1-11/+13
- provides the site packages fix - excludes the hdf5 linking changes (which are fixed in conduit@develop's build system) - relaxes constraints to allows building static ascent against shared python