summaryrefslogtreecommitdiff
path: root/share
AgeCommit message (Collapse)AuthorFilesLines
2023-03-15Hotfix: CI: Add CI target for gpu-test stack (#36136)kwryankrattiger1-0/+1
2023-03-14Fix typo affecting Gitlab CI (#36103)Harmen Stoppels1-1/+1
Introduced in #35944
2023-03-14concretizer: add mode to reuse dependencies only (#30990)Michael Kuhn1-9/+9
This adds a new mode for `concretizer:reuse` called `dependencies`, which only reuses dependencies. Currently, `spack install foo` will reuse older versions of `foo`, which might be surprising to users.
2023-03-13Set build_jobs dynamically in CI to avoid oversubscription (#35996)Harmen Stoppels4-2/+7
Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com> Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2023-03-10get --dev and drop set -x (#36010)Harmen Stoppels1-1/+1
2023-03-10curl flag is not universally supported (#36009)Harmen Stoppels3-3/+3
2023-03-10hotfix: fix double double quotes (#36005)Harmen Stoppels1-1/+1
2023-03-10ci: version bump for ghcr.io/spack/e4s-amazonlinux-2 (#35976)Zack Galbreath10-15/+15
* ci: version bump for ghcr.io/spack/e4s-amazonlinux-2 This new image comes with GnuPG v2.4.0 * py-cython: upperbounds for Python versions * fix py-gevent nonsense --------- Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-03-10CI boilerplate reduction (#34272)kwryankrattiger30-2527/+573
* CI configuration boilerplate reduction and refactor Configuration: - New notation for list concatenation (prepend/append) - New notation for string concatenation (prepend/append) - Break out configuration files for: ci.yaml, cdash.yaml, view.yaml - Spack CI section refactored to improve self-consistency and composability - Scripts are now lists of lists and/or lists of strings - Job attributes are now listed under precedence ordered list that are composed/merged using Spack config merge rules. - "service-jobs" are identified explicitly rather than as a batch CI: - Consolidate common, platform, and architecture configurations for all CI stacks into composable configuration files - Make padding consistent across all stacks (256) - Merge all package -> runner mappings to be consistent across all stacks Unit Test: - Refactor CI module unit-tests for refactor configuration Docs: - Add docs for new notations in configuration.rst - Rewrite docs on CI pipelines to be consistent with refactored CI workflow * Script verbose environ, dev bootstrap * Port #35409
2023-03-10ci: Increase the amount of pruning possible for PR pipelines (#35944)Scott Wittenburg1-0/+1
By setting the traversal depth to 1, only specs matching the changed package and direct dependents of those (and of course all dependencies of that set) are removed from pruning candidacy.
2023-03-03CI: Fix timeout for VisIt and ParaView (#35787)kwryankrattiger1-0/+2
ref. #35400
2023-03-01Remove handling of deprecated target names (graviton) (#35537)Massimiliano Culpo3-3/+3
* Update target names for Gitlab pipelines * Remove handling of deprecated names for graviton
2023-02-16Style: black 23, skip magic trailing comma (#35351)Adam J. Stewart1-11/+4
* Style: black 23, skip magic trailing commas * isort should use same line length as black * Fix unused import * Update version of black used in CI * Update new packages * Update new packages
2023-02-13ci: gpu test stack: turn off cuda@11.7 builds (#35467)eugeneswalker1-10/+0
2023-02-09gpu test stack: add test w/ ^cuda@12 builds on A100 w/ newer driver (#35375)eugeneswalker1-8/+13
* gpu test stack: test cuda@12 builds on A100 w/ newer driver * get gpu info via nvidia-smi; * kokkos+cuda^cuda@12 has genuine failure
2023-02-03ci: add minimal gpu testing stack (#35251)eugeneswalker3-0/+368
* ci: add minimal gpu testing stack * kokkos +cuda requires +wrapper... * require pass * add raja+cuda
2023-01-30e4s: add and prefer boost +python +filesystem +iostreams +system (#35180)eugeneswalker1-0/+3
2023-01-29e4s oneapi: use oneapi@2023.0.0 (#34634)eugeneswalker2-83/+98
2023-01-29e4s: add power stack (#32710)eugeneswalker3-317/+477
* e4s: restore builds builds * gitlab ci: allow UO to build protected binaries for signing * use newer image; comment out failing builds * gitlab-ci: Some tweaks for e4s power builds - fix tags (no longer require generate jobs to run on aws) - fix resource requests for generation jobs resource requests - remove SPACK_SIGNING_KEY from protected power build jobs - update UO signing key path - change the CDash build group to reflect stack name - retry pipeline generation jobs *always* * correct double packages: section * gitlab-ci:script: modernize * remove new gnu make, not for ppc64le --------- Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
2023-01-23e4s: add cusz +cuda (#35004)eugeneswalker1-0/+1
2023-01-21Add --exclude option to 'spack external find' (#35013)Alex Richert1-1/+1
* Add --exclude option to 'spack external find' to ignore user-specified external packages * Update bash completion arg order for external find Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2023-01-20Allocate more memory for generate jobs in all stacks (#35040)Zack Galbreath1-11/+2
* Allocate more memory for generate jobs in all stacks * Add a 60 minute timeout on generate jobs
2023-01-20e4s: ecp-data-vis-sdk +rocm: remove redundant amdgpu_target specification ↵eugeneswalker1-1/+1
(#35022)
2023-01-19depfile: --make-target-prefix -> --make-prefix (#35009)Harmen Stoppels1-1/+1
Since SPACK_PACKAGE_IDS is now also "namespaced" with <prefix>, it makes more sense to call the flag `--make-prefix` and alias the old flag `--make-target-prefix` to it.
2023-01-18e4s ci: clean up unnecessary constraints (#35006)eugeneswalker1-17/+11
2023-01-18e4s: add sz3 (#35007)eugeneswalker1-0/+1
2023-01-18e4s ci: add veloc (#35003)eugeneswalker1-0/+1
2023-01-18license year bump (#34921)Harmen Stoppels23-23/+23
* license bump year * fix black issues of modified files * mypy * fix 2021 -> 2023
2023-01-18e4s: add py-h5py ~mpi variant (#34998)eugeneswalker1-1/+2
2023-01-18depfile: variable with all identifiers (#34678)Harmen Stoppels1-0/+3
With the new variable [prefix/]SPACK_PACKAGE_IDS you can conveniently execute things after each successful install. For example push just-built packages to a buildcache ``` SPACK ?= spack export SPACK_COLOR = always MAKEFLAGS += -Orecurse MY_BUILDCACHE := $(CURDIR)/cache .PHONY: all clean all: push ifeq (,$(filter clean,$(MAKECMDGOALS))) include env.mk endif # the relevant part: push has *all* example/push/<pkg identifier> as prereqs push: $(addprefix example/push/,$(example/SPACK_PACKAGE_IDS)) $(SPACK) -e . buildcache update-index --directory $(MY_BUILDCACHE) $(info Pushed everything, yay!) # and each example/push/<pkg identifier> has the install target as prereq, # and the body can use target local $(HASH) and $(SPEC) variables to do # things, such as pushing to a build cache example/push/%: example/install/% @mkdir -p $(dir $@) $(SPACK) -e . buildcache create --allow-root --only=package --unsigned --directory $(MY_BUILDCACHE) /$(HASH) # push $(SPEC) @touch $@ spack.lock: spack.yaml $(SPACK) -e . concretize -f env.mk: spack.lock $(SPACK) -e . env depfile -o $@ --make-target-prefix example clean: rm -rf spack.lock env.mk example/ ``
2023-01-16Refer to mirrors by name, path, or url (#34891)Harmen Stoppels1-3/+13
With this change we get the invariant that `mirror.fetch_url` and `mirror.push_url` return valid URLs, even when the backing config file is actually using (relative) paths with potentially `$spack` and `$env` like variables. Secondly it avoids expanding mirror path / URLs too early, so if I say `spack mirror add name ./path`, it stays `./path` in my config. When it's retrieved through MirrorCollection() we exand it to say `file://<env dir>/path` if `./path` was set in an environment scope. Thirdly, the interface is simplified for the relevant buildcache commands, so it's more like `git push`: ``` spack buildcache create [mirror] [specs...] ``` `mirror` is either a mirror name, a path, or a URL. Resolving the relevant mirror goes as follows: - If it contains either / or \ it is used as an anonymous mirror with path or url. - Otherwise, it's interpreted as a named mirror, which must exist. This helps to guard against typos, e.g. typing `my-mirror` when there is no such named mirror now errors with: ``` $ spack -e . buildcache create my-mirror ==> Error: no mirror named "my-mirror". Did you mean ./my-mirror? ``` instead of creating a directory in the current working directory. I think this is reasonable, as the alternative (requiring that a local dir exists) feels a bit pendantic in the general case -- spack is happy to create the build cache dir when needed, saving a `mkdir`. The old (now deprecated) format will still be available in Spack 0.20, but is scheduled to be removed in 0.21: ``` spack buildcache create (--directory | --mirror-url | --mirror-name) [specs...] ``` This PR also touches `tmp_scope` in tests, because it didn't really work for me, since spack fixes the possible --scope values once and for all across tests, so tests failed when run out of order.
2023-01-16e4s: add py-h5py (#34914)eugeneswalker1-0/+1
2023-01-16e4s ci: py-libensemble: activate variants (#34915)eugeneswalker1-1/+1
2023-01-14spack list: add `--count` option (#34950)Todd Gamblin1-1/+1
Sometimes I just want to know how many packages of a certain type there are. - [x] add `--count` option to `spack list` that output the number of packages that *would* be listed. ```console > spack list --count 6864 > spack list --count py- 2040 > spack list --count r- 1162 ```
2023-01-13Paraview rocm (#34790)kwryankrattiger1-4/+7
* paraview: add `rocm` variant This conflicts with CUDA and requires at least ParaView 5.11.0. More dependencies are also needed. * E4S: Add ParaView for ROCm and CUDA stacks * DAV SDK: Update ParaView version and GPU variants * Verify using hipcc vs amdclang++ for newer hip Co-authored-by: Ben Boeckel <ben.boeckel@kitware.com>
2023-01-11Bump memory requests for e4s/ml generate jobs (#34648)Mike VanDenburgh1-0/+10
2023-01-10protected runner jobs at uo: trust e4s-uo signing key (#34828)eugeneswalker12-0/+36
2023-01-10gitlab ci: fix tags for pipeline generation jobs (#34789)Scott Wittenburg1-37/+44
Gitlab does not merge lists when a job extends two other definitions that include the same list (e.g. tags). Also, it merges dictionaries as long as the keys are distinct, but just takes the last mentioned value when there are key collisions. This change makes sure that when different tags are needed by a pipeline, the ones we want are actually provided. It also changes the example stack to better follow this pattern so we do not lead developers astray in the future.
2023-01-10Add CUDA 12.0 (#34664)Wileam Y. Phan1-4/+4
2023-01-10e4s ci: use 2023-01-01 runner image (#34827)eugeneswalker2-4/+4
2023-01-04e4s: move default values for rocm/cuda arch into packags:all:variants (#34772)Harmen Stoppels1-65/+58
2022-12-27spack graph: rework to use Jinja templates and builders (#34637)Massimiliano Culpo2-1/+34
`spack graph` has been reworked to use: - Jinja templates - builder objects to construct the template context when DOT graphs are requested. This allowed to add a new colored output for DOT graphs that highlights both the dependency types and the nodes that are needed at runtime for a given spec.
2022-12-22ML CI: Linux x86_64 (#34299)Adam J. Stewart4-256/+283
* ML CI: Linux x86_64 * Update comments * Rename again * Rename comments * Update to match other arches * No compiler * Compiler was wrong anyway * Faster TF
2022-12-17e4s: disable mac stack due to binary relocation issue#32571 (#34560)eugeneswalker1-103/+103
2022-12-15gitlab ci: more resources for slow builds (#34505)Zack Galbreath8-2/+28
2022-12-13Use file paths/urls correctly (#34452)Harmen Stoppels1-1/+1
The main issue that's fixed is that Spack passes paths (as strings) to functions that require urls. That wasn't an issue on unix, since there you can simply concatenate `file://` and `path` and all is good, but on Windows that gives invalid file urls. Also on Unix, Spack would not deal with uri encoding like x%20y for file paths. It also removes Spack's custom url.parse function, which had its own incorrect interpretation of file urls, taking file://x/y to mean the relative path x/y instead of hostname=x and path=/y. Also it automatically interpolated variables, which is surprising for a function that parses URLs. Instead of all sorts of ad-hoc `if windows: fix_broken_file_url` this PR adds two helper functions around Python's own path2url and reverse. Also fixes a bug where some `spack buildcache` commands used `-d` as a flag to mean `--mirror-url` requiring a URL, and others `--directory`, requiring a path. It is now the latter consistently.
2022-12-09gitlab ci: more resources for paraview and py-torch (#34412)Scott Wittenburg4-1/+6
2022-12-09new command: `spack pkg grep` to search package files (#34388)Todd Gamblin2-2/+12
It's very common for us to tell users to grep through the existing Spack packages to find examples of what they want, and it's also very common for package developers to do it. Now, searching packages is even easier. `spack pkg grep` runs grep on all `package.py` files in repos known to Spack. It has no special options other than the search string; all options passed to it are forwarded along to `grep`. ```console > spack pkg grep --help usage: spack pkg grep [--help] ... positional arguments: grep_args arguments for grep options: --help show this help message and exit ``` ```console > spack pkg grep CMakePackage | head -3 /Users/gamblin2/src/spack/var/spack/repos/builtin/packages/3dtk/package.py:class _3dtk(CMakePackage): /Users/gamblin2/src/spack/var/spack/repos/builtin/packages/abseil-cpp/package.py:class AbseilCpp(CMakePackage): /Users/gamblin2/src/spack/var/spack/repos/builtin/packages/accfft/package.py:class Accfft(CMakePackage, CudaPackage): ``` ```console > spack pkg grep -Eho '(\S*)\(PythonPackage\)' | head -3 AwsParallelcluster(PythonPackage) Awscli(PythonPackage) Bueno(PythonPackage) ``` ## Return Value This retains the return value semantics of `grep`: * 0 for found, * 1 for not found * >1 for error ## Choosing a `grep` You can set the ``SPACK_GREP`` environment variable to choose the ``grep`` executable this command should use.
2022-12-09Windows: reenable unit tests (#33385)John W. Parent4-14/+10
Unit tests on Windows are supposed to pass for any PR to pass CI. However, the return code for the unit test command was not being checked, which meant this check was always passing (effectively disabled). This PR * Properly checks the result of the unit tests and fails if the unit tests fail * Fixes (or disables on Windows) a number of tests which have "drifted" out of support on Windows since this check was effectively disabled
2022-12-09bugfix: `spack load` shell test can fail on macos (#34419)Todd Gamblin1-1/+2
At some point the `a` mock package became an `AutotoolsPackage`, and that means it depends on `gnuconfig` on macOS. This was causing one of our shell tests to fail on macOS because it was testing for `{a.prefix.bin}:{b.prefix.bin}` in `PATH`, but `gnuconfig` shows up between them. - [x] simplify the test to check `spack load --sh a` and `spack load --sh b` separately
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2024-11-10 10:15:16 +0000