summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2023-10-27e4s ci stacks: add exago specs (#40712)eugeneswalker4-1/+7
* e4s ci: add exago +cuda, +rocm builds * exago: rename 5-18-2022-snapshot to snapshot.5-18-2022 * disable exago +rocm for non-external rocm ci install * note that hiop +rocm fails to find hip libraries when they are spack-installed
2023-10-27mgard@2020-10-01 %oneapi@2023: turn of c++11-narrowing via cxxflags (#40743)eugeneswalker1-0/+6
2023-10-27External finding: update default paths; treat .bat as executable on Windows ↵John W. Parent4-82/+89
(#39850) .bat or .exe files can be considered executable on Windows. This PR expands the regex for detectable packages to allow for the detection of packages that vendor .bat wrappers (intel mpi for example). Additional changes: * Outside of Windows, when searching for executables `path_hints=None` was used to indicate that default path hints should be provided, and `[]` was taken to mean that no defaults should be chosen (in that case, nothing is searched); behavior on Windows has now been updated to match. * Above logic for handling of `path_hints=[]` has also been extended to library search (for both Linux and Windows). * All exceptions for external packages were documented as timeout errors: this commit adds a distinction for other types of errors in warning messages to the user.
2023-10-27OCI buildcache (#38358)Harmen Stoppels24-242/+3479
Credits to @ChristianKniep for advocating the idea of OCI image layers being identical to spack buildcache tarballs. With this you can configure an OCI registry as a buildcache: ```console $ spack mirror add my_registry oci://user/image # Dockerhub $ spack mirror add my_registry oci://ghcr.io/haampie/spack-test # GHCR $ spack mirror set --push --oci-username ... --oci-password ... my_registry # set login credentials ``` which should result in this config: ```yaml mirrors: my_registry: url: oci://ghcr.io/haampie/spack-test push: access_pair: [<username>, <password>] ``` It can be used like any other registry ``` spack buildcache push my_registry [specs...] ``` It will upload the Spack tarballs in parallel, as well as manifest + config files s.t. the binaries are compatible with `docker pull` or `skopeo copy`. In fact, a base image can be added to get a _runnable_ image: ```console $ spack buildcache push --base-image ubuntu:23.04 my_registry python Pushed ... as [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack $ docker run --rm -it [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack ``` which should really be a game changer for sharing binaries. Further, all content-addressable blobs that are downloaded and verified will be cached in Spack's download cache. This should make repeated `push` commands faster, as well as `push` followed by a separate `update-index` command. An end to end example of how to use this in Github Actions is here: **https://github.com/haampie/spack-oci-buildcache-example** TODO: - [x] Generate environment modifications in config so PATH is set up - [x] Enrich config with Spack's `spec` json (this is allowed in the OCI specification) - [x] When ^ is done, add logic to create an index in say `<image>:index` by fetching all config files (using OCI distribution discovery API) - [x] Add logic to use object storage in an OCI registry in `spack install`. - [x] Make the user pick the base image for generated OCI images. - [x] Update buildcache install logic to deal with absolute paths in tarballs - [x] Merge with `spack buildcache` command - [x] Merge #37441 (included here) - [x] Merge #39077 (included here) - [x] #39187 + #39285 - [x] #39341 - [x] Not a blocker: #35737 fixes correctness run env for the generated container images NOTE: 1. `oci://` is unfortunately taken, so it's being abused in this PR to mean "oci type mirror". `skopeo` uses `docker://` which I'd like to avoid, given that classical docker v1 registries are not supported. 2. this is currently `https`-only, given that basic auth is used to login. I _could_ be convinced to allow http, but I'd prefer not to, given that for a `spack buildcache push` command multiple domains can be involved (auth server, source of base image, destination registry). Right now, no urllib http handler is added, so redirects to https and auth servers with http urls will simply result in a hard failure. CAVEATS: 1. Signing is not implemented in this PR. `gpg --clearsign` is not the nicest solution, since (a) the spec.json is merged into the image config, which must be valid json, and (b) it would be better to sign the manifest (referencing both config/spec file and tarball) using more conventional image signing tools 2. `spack.binary_distribution.push` is not yet implemented for the OCI buildcache, only `spack buildcache push` is. This is because I'd like to always push images + deps to the registry, so that it's `docker pull`-able, whereas in `spack ci` we really wanna push an individual package without its deps to say `pr-xyz`, while its deps reside in some `develop` buildcache. 3. The `push -j ...` flag only works for OCI buildcache, not for others
2023-10-27octopus: split netcdf-c and netcdf-fortran dependency (#40685)Ashwin Kumar Karnad1-2/+3
2023-10-27strumpack: add version 7.2.0 (#40732)Satish Balay1-0/+1
2023-10-27gromacs: default to external blas & lapack (#40490)Harmen Stoppels1-28/+8
* gromacs: default to external blas & lapack * drop vendored lapack/blas altogether
2023-10-27ci: spack compiler find should list extra config scopes (#40727)Harmen Stoppels1-1/+6
otherwise it detected pre-configured compilers in an potentially different way.
2023-10-27Fixes and options for package spglib (#40684)dmt41-1/+6
* Fix cmake_args for spglib v2.1.0+ * Add option to build fortran interface in package spglib * fix style as sugested by ci/prechecks/style * Enable fortran variant from v1.16.4 as suggested Co-authored-by: Rocco Meli <r.meli@bluemail.ch> --------- Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
2023-10-27gromacs: fix version branch in intel fftw (#40489)Harmen Stoppels1-1/+1
2023-10-27akantu: use f-strings (#40466)Alec Scott1-6/+4
Co-authored-by: Nicolas Richart <nrichart@users.noreply.github.com>
2023-10-26plasma: add version 23.8.2 (#40728)Satish Balay1-0/+2
2023-10-26itk: misc fixes (#39832)Carlos Bederián1-2/+7
* itk: patch missing include for newer compilers * itk: The package doesn't use MPI * itk: package requires the high-level hdf5 api * itk: patch url with ?full_index=1 * itk: point to 4041 commit in master * itk: don't constrain hdf5 with ~mpi
2023-10-26ldak: add v5.2 & add maintainer (#40710)snehring1-18/+55
* ldak: update to 5.2, add maintainer * ldak: use compiler.openmp_flag
2023-10-26dataTransferKit: add v3.1.1, v3.1.0 (#40556)Daniel Arndt1-2/+5
* Update DataTransferKit for 3.1.1 release * Require Trilinos-14 for 3.1.0 and higher
2023-10-26PythonPackage: allow archive_files to be overridden (#40694)Adam J. Stewart1-1/+1
2023-10-26curl: Fix librtmp variant (#40713)Torbjörn Lönnemark7-0/+394
* rtmpdump: New package * curl: Fix librtmp variant Add the previously missing dependency required for rtmp support. The variant has been broken since its addition in PR #25166. Fixes one of the two issues reported in #26887.
2023-10-26unmaintained a* packages: update to use f-strings (#40467)Alec Scott42-164/+137
2023-10-26Update spack package for exago@1.6.0 release (#40614)Ryan Danehy1-16/+34
* Update spack package for exago:1.6.0 * update style * Weird spack style env bug fixed * Update spack package for exago:1.6.0 * update style * Weird spack style env bug fixed * changes to allow release 1.6.0 * fix depends, and versioning * rm cmake variable * add s * style fix --------- Co-authored-by: Ryan Danehy <dane678@deception04.pnl.gov> Co-authored-by: Ryan Danehy <dane678@deception03.pnl.gov> Co-authored-by: ryan.danehy@pnnl.gov <dane678@we45149.home>
2023-10-26spack checksum pkg@1.2, use as version filter (#39694)Harmen Stoppels3-25/+15
* spack checksum pkg@1.2, use as version filter Currently pkg@1.2 splits on @ and looks for 1.2 specifically, with this PR pkg@1.2 is a filter so any matching 1.2, 1.2.1, ..., 1.2.10 version is displayed. * fix tests * fix style
2023-10-26Add conflict between cxxstd > 17 and cuda < 12 in pika (#40717)Auriane R1-2/+4
* Add conflict with C++ standard > 17 and cuda < 12 * Removing map_cxxstd since boost supports C++20 flag
2023-10-26modules: no --delim option if separator is colon character (#39010)Xavier Delaruelle2-25/+23
Update Tcl modulefile template to simplify generated `append-path`, `prepend-path` and `remove-path` commands and improve their readability. If path element delimiter is colon character, do not set the `--delim` option as it is the default delimiter value.
2023-10-26PythonPackage: nested config_settings (#40693)Adam J. Stewart1-17/+26
* PythonPackage: nested config_settings * flake8
2023-10-26spack checksum: show long flags in usage output (#40407)Harmen Stoppels3-13/+13
2023-10-26modules: hide implicit modulefiles (#36619)Xavier Delaruelle10-20/+407
Renames exclude_implicits to hide_implicits When hide_implicits option is enabled, generate modulefile of implicitly installed software and hide them. Even if implicit, those modulefiles may be referred as dependency in other modulefiles thus they should be generated to make module properly load dependent module. A new hidden property is added to BaseConfiguration class. To hide modulefiles, modulercs are generated along modulefiles. Such rc files contain specific module command to indicate a module should be hidden (for instance when using "module avail"). A modulerc property is added to TclFileLayout and LmodFileLayout classes to get fully qualified path name of the modulerc associated to a given modulefile. Modulerc files will be located in each module directory, next to the version modulefiles. This scheme is supported by both module tool implementations. modulerc_header and hide_cmd_format attributes are added to TclModulefileWriter and LmodModulefileWriter. They help to know how to generate a modulerc file with hidden commands for each module tool. Tcl modulerc file requires an header. As we use a command introduced on Modules 4.7 (module-hide --hidden-loaded), a version requirement is added to header string. For lmod, modules that open up a hierarchy are never hidden, even if they are implicitly installed. Modulerc is created, updated or removed when associated modulefile is written or removed. If an implicit modulefile becomes explicit, hidden command in modulerc for this modulefile is removed. If modulerc becomes empty, this file is removed. Modulerc file is not rewritten when no content change is detected. Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-10-26libluv: require CMake 3 and CMP0042 (#40716)Alberto Invernizzi2-3/+7
2023-10-26git versions: fix commit shas [automated] (#40703)Harmen Stoppels8-22/+54
2023-10-25Paraview 5.12 prep (#40527)Ben Boeckel2-3/+241
* paraview: rebase the adios2 patch for 5.12-to-be * paraview: disable fastfloat and token for 5.12-to-be * paraview: require older protobuf for 5.12 as well * paraview: require C++11-supporting protobuf for `master` too
2023-10-25fdb: add releases v5.11.23 and v5.11.17 (#40571)Victoria Cherkas1-0/+3
2023-10-25initial commit to fix mivisionx build for 5.6 (#40579)afzpatel1-1/+1
2023-10-25cuda: add NVHPC_CUDA_HOME. (#40507)Dominic Hofer1-0/+3
* [cuda] Add NVHPC_CUDA_HOME. * Add CUDA_HOME and NVHC_CUDA_HOME to cuda's dependent build env. --------- Co-authored-by: Dominic Hofer <dominic.hofer@meteoswiss.ch>
2023-10-25Added Highway versions up to 1.0.7 (#40691)Filippo Barbari1-0/+6
2023-10-25PyTorch: patch breakpad dependency (#40648)Adam J. Stewart1-1/+9
2023-10-25build(deps): bump pytest from 7.4.2 to 7.4.3 in /lib/spack/docs (#40697)dependabot[bot]1-1/+1
2023-10-25ci: don't put compilers in config (#40700)Harmen Stoppels8-89/+7
* ci: don't register detectable compilers Cause they go out of sync... * remove intel compiler, it can be detected too * Do not run spack compiler find since compilers are registered in concretize job already * trilinos: work around +stokhos +cuda +superlu-dist bug due to EMPTY macro
2023-10-25ci: darwin aarch64 use apple-clang-15 tag (#40706)Harmen Stoppels2-2/+2
2023-10-25py-lightning: py-torch~distributed is broken again (#40696)Adam J. Stewart1-0/+3
2023-10-25Add dlaf variant to cp2k (#40702)Rocco Meli1-0/+17
2023-10-25[cp2k] Use fftw3 MKL by default when cp2k is compiled with mkl (#40671)Taillefumier Mathieu1-20/+134
2023-10-24Windows: search PATH for patch utility (#40513)John W. Parent1-4/+7
Previously, we only searched for `patch` inside of whatever Git installation was available because the most common installation of Git available on Windows had `patch`. That's not true for all possible installations of Git though, so this updates the search to also check PATH.
2023-10-24Add ufs-utils@1.11.0 (#40695)Alex Richert1-0/+6
* Add ufs-utils@1.11.0 * Update package.py
2023-10-24Updating rvs binary path. (#40604)renjithravindrankannath1-2/+6
* Updating rvs binary path * Updating spec check as per the recommendation
2023-10-24openmpi: fix pmi@4.2.3: compat (#40686)AMD Toolchain Support1-5/+8
2023-10-24Added new benchmark version up to 1.8.3 (#40689)Filippo Barbari1-0/+9
2023-10-24neovim: conflict for libluv problem on macOS + add newer versions of neovim ↵Alberto Invernizzi2-1/+8
and libluv (#40690) * add conflict with libluv version >=1.44 just on macOS * minor change * add libluv versions * neovim: add newer releases
2023-10-24hiop +cuda: fix issue 40678 (#40688)eugeneswalker1-3/+2
2023-10-24exago: fix v1.5.1 tag; only allow python up to 3.10 for for @:1.5 (#40676)eugeneswalker2-2/+6
* exago: fix v1.5.1 tag; only allow python up to 3.10 for for @:1.5 due to pybind error with py 3.11 * hiop@:1.0 +cuda: constrain to cuda@:11.9
2023-10-24armpl-gcc: add version 23.10 and macOS support (#40511)Annop Wongwathanarat1-15/+104
2023-10-23fix installation error of bear (#40637)Nakano Masaki3-4/+9
Co-authored-by: Tom Scogland <scogland1@llnl.gov>
2023-10-23Adios2: add kokkos variant (#40623)Vicente Bolea5-9/+59
* adios2: update variants and dependencies * adios2: add kokkos rocm|cuda|sycl variant * e4s oneapi ci stack: add adios2 +sycl * e4s ci stack: add adios2 +rocm * [@spackbot] updating style on behalf of vicentebolea * Apply suggestions from code review * adios2: fixed cuda variant * update ecp-data-vis-sdk * Update share/spack/gitlab/cloud_pipelines/stacks/e4s-power/spack.yaml --------- Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com> Co-authored-by: vicentebolea <vicentebolea@users.noreply.github.com>