summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2023-10-19Improve setup build / run / test environment (#35737)Harmen Stoppels22-416/+565
This adds a `SetupContext` class which is responsible for setting package.py module globals, and computing the changes to environment variables for the build, test or run context. The class uses `effective_deptypes` which takes a list of specs (e.g. single item of a spec to build, or a list of environment roots) and a context (build, run, test), and outputs a flat list of specs that affect the environment together with a flag in what way they do so. This list is topologically ordered from root to leaf, so that one can be assured that dependents override variables set by dependencies, not the other way around. This is used to replace the logic in `modifications_from_dependencies`, which has several issues: missing calls to `setup_run_environment`, and the order in which operations are applied. Further, it should improve performance a bit in certain cases, since `effective_deptypes` run in O(v + e) time, whereas `spack env activate` currently can take up to O(v^2 + e) time due to loops over roots. Each edge in the DAG is visited once by calling `effective_deptypes` with `env.concrete_roots()`. By marking and propagating flags through the DAG, this commit also fixes a bug where Spack wouldn't call `setup_run_environment` for runtime dependencies of link dependencies. And this PR ensures that Spack correctly sets up the runtime environment of direct build dependencies. Regarding test dependencies: in a build context they are are build-time test deps, whereas in a test context they are install-time test deps. Since there are no means to distinguish the build/install type test deps, they're both. Further changes: - all `package.py` module globals are guaranteed to be set before any of the `setup_(dependent)_(run|build)_env` functions is called - traversal order during setup: first the group of externals, then the group of non-externals, with specs in each group traversed topological (dependencies are setup before dependents) - modules: only ever call `setup_dependent_run_environment` of *direct* link/run type deps - the marker in `set_module_variables_for_package` is dropped, since we should call the method once per spec. This allows us to set only a cheap subset of globals on the module: for example it's not necessary to compute the expensive `cmake_args` and w/e if the spec under consideration is not the root node to be built. - `spack load`'s `--only` is deprecated (it has no effect now), and `spack load x` now means: do everything that's required for `x` to work at runtime, which requires runtime deps to be setup -- just like `spack env activate`. - `spack load` no longer loads build deps (of build deps) ... - `spack env activate` on partially installed or broken environments: this is all or nothing now. If some spec errors during setup of its runtime env, you'll only get the unconditional variables + a warning that says the runtime changes for specs couldn't be applied. - Remove traversal in upward direction from `setup_dependent_*` in packages. Upward traversal may iterate to specs that aren't children of the roots (e.g. zlib / python have hundreds of dependents, only a small fraction is reachable from the roots. Packages should only modify the direct dependent they receive as an argument)
2023-10-19spack checksum: restore ability to select top n (#40531)Harmen Stoppels2-6/+40
The ability to select the top N versions got removed in the checksum overhaul, cause initially numbers were used for commands. Now that we settled on characters for commands, let's make numbers pick the top N again.
2023-10-19Automated deployment to update package flux-core 2023-10-19 (#40605)Vanessasaurus1-0/+1
Co-authored-by: github-actions <github-actions@users.noreply.github.com>
2023-10-19libvorbis: drop -force_cpusubtype_ALL flag (#40616)Harmen Stoppels1-0/+3
This flag was only relevant when targeting powerpc from apple-clang, which we don't do. The flag is removed from apple-clang@15. Let's drop it unconditionally.
2023-10-19gitlab ci: Rework how mirrors are configured (#39939)Scott Wittenburg36-150/+273
Improve how mirrors are used in gitlab ci, where we have until now thought of them as only a string. By configuring ci mirrors ahead of time using the proposed mirror templates, and by taking advantage of the expressiveness that spack now has for mirrors, this PR will allow us to easily switch the protocol/url we use for fetching binary dependencies. This change also deprecates some gitlab functionality and marks it for removal in Spack 0.23: - arguments to "spack ci generate": * --buildcache-destination * --copy-to - gitlab configuration options: * enable-artifacts-buildcache * temporary-storage-url-prefix
2023-10-19petsc: add variant +sycl (#40562)Satish Balay1-0/+16
* petsc: add variant +sycl * petsc: add in gmake as dependency - so that consistent make gets used between petsc and slepc builds [that can have different env for each of the builds]
2023-10-19ci: remove incorrect compilers.yaml (#40610)Harmen Stoppels1-27/+0
2023-10-19ASP-based solver: single Spec instance per dag hash (#39590)Massimiliano Culpo2-32/+127
Reused specs used to be referenced directly into the built spec. This might cause issues like in issue 39570 where two objects in memory represent the same node, because two reused specs were loaded from different sources but referred to the same spec by DAG hash. The issue is solved by copying concrete specs to a dictionary keyed by dag hash.
2023-10-19Stand-alone test feature deprecation postponed to v0.22 (#40600)Tamara Dahlgren1-2/+2
2023-10-19byte-unixbench: respect compiler choice (#39242)Aiden Grossman1-0/+4
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2023-10-19connect-proxy: respect compiler choice (#39243)Aiden Grossman1-0/+3
2023-10-19bioawk: respect compiler choice (#39241)Aiden Grossman1-0/+3
2023-10-19busybox: respect compiler choice (#39239)Aiden Grossman1-2/+2
2023-10-19julia: Fix build for @1.9 (#39045)Michael Kuhn1-0/+2
julia@1.9 tries to download ittapi, which requires cmake. Disable it explicitly.
2023-10-18acfl: add version 23.10 (#40510)Annop Wongwathanarat1-1/+35
2023-10-18unparse: also support generic type aliases (#40328)Harmen Stoppels1-0/+4
2023-10-18AutotoolsPackage / MakefilePackage: add gmake build dependency (#40380)Harmen Stoppels13-86/+133
2023-10-18llvm: fix ncurses+termlib linking in lldb (#40594)Harmen Stoppels1-0/+10
2023-10-18Fix dev-build keep_stage behavior (#40576)Harmen Stoppels2-52/+29
`spack dev-build` would incorrectly set `keep_stage=True` for the entire DAG, including for non-dev specs, even though the dev specs have a DIYStage which never deletes sources.
2023-10-18[fix] py-werkzeug: add constraint in python dependence (#40590)Lydéric Debusschère1-0/+1
py-werkzeug@:0.12 does not work with python@3.10: Test with py-werkzeug 0.12.2 and python 3.10: ``` $ python3.10 -c 'import werkzeug' py-werkzeug-0.12.2/lib/python3.11/site-packages/werkzeug/datastructures.py", line 16, in <module> from collections import Container, Iterable, MutableSet ImportError: cannot import name 'Container' from 'collections' ``` Test with py-werkzeug 0.12.2 and python 3.9: ``` python3.9 -c "from collections import Container" <string>:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working ```
2023-10-18Add license directive (#39346)Aiden Grossman9-0/+154
This patch adds in a license directive to get the ball rolling on adding in license information about packages to spack. I'm primarily interested in just adding license into spack, but this would also help with other efforts that people are interested in such as adding license information to the ASP solve for concretization to make sure licenses are compatible. Usage: Specifying the specific license that a package is released under in a project's `package.py` is good practice. To specify a license, find the SPDX identifier for a project and then add it using the license directive: ```python license("<SPDX Identifier HERE>") ``` For example, for Apache 2.0, you might write: ```python license("Apache-2.0") ``` Note that specifying a license without a when clause makes it apply to all versions and variants of the package, which might not actually be the case. For example, a project might have switched licenses at some point or have certain build configurations that include files that are licensed differently. To account for this, you can specify when licenses should be applied. For example, to specify that a specific license identifier should only apply to versionup to and including 1.5, you could write the following directive: ```python license("MIT", when="@:1.5") ```
2023-10-18abi.py: fix typo, add type-hints (#38216)Greg Becker2-13/+20
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-10-17paraview: add variant for NVIDIA IndeX (#40577)jfavre1-0/+4
* add variant for NVIDIA IndeX * remove whitespaces
2023-10-17xmlto: add more dependencies (#40578)Dennis Klein1-1/+14
`xmllint` is called by `xmlto` during generation of `libzmq`'s docs, so adding `libxml2`. The docbook deps and the patches are taken from https://src.fedoraproject.org/rpms/xmlto/blob/rawhide/f/xmlto.spec There are still many more dependencies missing, but this is out of scope of this patch (which is only concerned about the use case of `libzmq`).
2023-10-17Add tracy 0.10 (#40573)Mikael Simberg2-0/+2
2023-10-17Allow / in GitVersion (#39398)Peter Scheibel16-27/+214
This commit allows version specifiers to refer to git branches that contain forward slashes. For example, the following is valid syntax now: pkg@git.releases/1.0 It also adds a new method `Spec.format_path(fmt)` which is like `Spec.format`, but also maps unsafe characters to `_` after interpolation. The difference is as follows: >>> Spec("pkg@git.releases/1.0").format("{name}/{version}") 'pkg/git.releases/1.0' >>> Spec("pkg@git.releases/1.0").format_path("{name}/{version}") 'pkg/git.releases_1.0' The `format_path` method is used in all projections. Notice that this method also maps `=` to `_` >>> Spec("pkg@git.main=1.0").format_path("{name}/{version}") 'pkg/git.main_1.0' which should avoid syntax issues when `Spec.prefix` is literally copied into a Makefile as sometimes happens in AutotoolsPackage or MakefilePackage
2023-10-17Add mpi_f08 variant to CP2K (#40574)Rocco Meli1-0/+9
* add mpi_f08 variant * add conflict * add conflict with released versions of cp2k and +mpi_f08
2023-10-17py-rtree: add v1.1.0 (#40575)Adam J. Stewart1-2/+3
2023-10-17hiop: 1.0.1 release (#40580)Cameron Rutherford1-0/+1
2023-10-17patchelf: fix compilation with GCC 7 (#40581)Harmen Stoppels2-0/+28
2023-10-17Support apple-clang in pdt (#40582)wspear1-2/+2
2023-10-17Support `spack env activate --with-view <name> <env>` (#40549)Harmen Stoppels7-75/+120
Currently `spack env activate --with-view` exists, but is a no-op. So, it is not too much of a breaking change to make this redundant flag accept a value `spack env activate --with-view <name>` which activates a particular view by name. The view name is stored in `SPACK_ENV_VIEW`. This also fixes an issue where deactivating a view that was activated with `--without-view` possibly removes entries from PATH, since now we keep track of whether the default view was "enabled" or not.
2023-10-17packages: use "requires" to allow only selected compilers (#40567)Massimiliano Culpo12-103/+34
A few packages have encoded an idiom that pre-dates the introduction of the 'requires' directive, and they cycle over all compilers to conflict with the ones that are not supported. Here instead we reverse the logic, and require the ones that are supported.
2023-10-16Creation of Beatnik package and associated updates to silo and cabana spack ↵Patrick Bridges3-10/+129
package (#40382) * Added initial package for building Beatnik with spack * Fixed github ID for Jason as a maintainer. * Major revision of beatnik spack package to properly support GPU spack builds with CUDA (and ROCm, though that it untested) * Marked that beatnik 1.0 will require cabana 0.6.0. We will wait for the cabana 0.6.0 release before we release beatnik * Update to beatnik package spec to compile with hipcc when +rocm * Updated spack package for cabana for version 0.6.0 and appropriate heffte dependency * Updated beatnik package to require cabana 0.6.0 * More updates to cabana and beatnik to build with cabana 0.6.0 * Finish removing BLT dependence from beatnik * More updates to beatnik package for compiling on cray systems * Updated beatnik package for new cabana package * Changes to silo package for new silo version * Fixed version specs for heffte to be able to concretize and build * Fixed spack style issues for beatnik and silo packages * More spack formatting fixes to beatnik and silo * Patrick adopting silo package as maintainer for now * Should address final style changes to beatnik package spec * Yet more style fixes. * Perhaps this is the final style fixes? :) * Minor fix to cabana package on required versions
2023-10-16Add latest versions of rlwrap (#40563)Eric Berquist1-0/+11
* Add latest versions of rlwrap * rlwrap: fix URL for v0.46.1
2023-10-16intel-oneapi-compilers: ifx is located in bin not bin/intel64 (#40561)Stephen Sachs1-1/+4
This is a fix on top of https://github.com/spack/spack/pull/40557 . Tagging @rscohn2 for review.
2023-10-16py-furo: new version (#40559)Seth R. Johnson1-0/+1
2023-10-16py-grayskull: add new package (#40293)Adam J. Stewart6-8/+111
* py-grayskull: add new package * [@spackbot] updating style on behalf of adamjstewart --------- Co-authored-by: adamjstewart <adamjstewart@users.noreply.github.com>
2023-10-16Use string representation of deptypes for concrete specs (#40566)Massimiliano Culpo1-1/+5
2023-10-16py-fiona: add v1.9.5 (#40497)Adam J. Stewart1-7/+18
2023-10-16py-shapely: add v2.0.2 (#40523)Adam J. Stewart1-1/+3
2023-10-16py-grpcio: cython 3 still not supported (#40537)Adam J. Stewart1-1/+1
2023-10-16py-lightning: add v2.1.0 (#40496)Adam J. Stewart1-46/+55
2023-10-16Updating patch to enable flag mcode-object-version=none (#40367)renjithravindrankannath6-30/+64
* Updating patch to add flag mcode-object-version=none when device libs is buils as part of llvm-amdgpu * Limiting patch to +rocm-device-libs variant and adding appropriate comment for the patch * Updating llvmpatch as per the mailine code Updating hsa-rocr patch as per the latest code Updating the if elif condition for the hip test src path * Updating flags for 5.5 relases and above * Updating build flags and patches
2023-10-16Fix version incompatibilities of py-pandas and py-openpyxl (#40472)Dom Heinzeller4-0/+57
* Fix version incompatibilities of py-pandas and py-openpyxl * Add variant excel for py-pandas * Add package py-pyxlsb * Add versios for py-xlsxwriter * Define excel dependencies for py-pandas 1.4, 1.5, 2.0, 2.1 * Fix variant excel in py-pandas * Add package py-odfpy, which is also a dependency for py-pandas@2.0: * Rearrange excel dependencies for py-pandas * Change url to pypi * Add missing newline to fix style in py-odfpy
2023-10-16Add nextflow 23.10.0 (#40547)Diego Alvarez S1-0/+5
2023-10-16fenics-basix: update for v0.7 (#40440)Garth N. Wells2-5/+10
* Uodate for Basix 0.7 * Version fix for nanobind dependency * Simplification * Version update and simplify dependencies * Add comment on location of pyproject.toml * Update var/spack/repos/builtin/packages/py-fenics-basix/package.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> --------- Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2023-10-16intel-oneapi-compilers: ifx uses --gcc-name & --gxx-name (#40557)Stephen Sachs1-2/+2
`ifx` uses the older syntax instead of `--gcc-toolchain`. Tested up to version 2023.2.0.
2023-10-16py-fenics-ufl: update version and add test (#40534)Garth N. Wells1-4/+15
* Update py-ufl vesion * Syntax fix * Syntax fix * Add test * Updates following comments
2023-10-16[add] py-cylc-uiserver: new recipe (#39983)Lydéric Debusschère1-0/+32
* [add] py-cylc-uiserver: new recipe * py-cylc-uiserver: remove version constraint on the dependence python * [fix] py-cylc-uiserver: add forgotten dependence py-graphql-core --------- Co-authored-by: LydDeb <lyderic.debusschere.tgcc@cea.fr>