summaryrefslogtreecommitdiff
path: root/share
AgeCommit message (Collapse)AuthorFilesLines
2023-08-07CI: remove redundant sections (#38514)kwryankrattiger9-716/+70
Refactor gitlab ci configs so that mac and cray jobs can reuse as much higher level configuration as possible. * CI: remove redundant sections * CI: Include base linux CI configs in cray stacks Relocation and runner mapping is consistent between cray and linux runners. * Export user cache path in before script * CI: add GPG root for mac runners * Disable user configs Metal runners share a ~ directory * Disable user config and add configs in activate env
2023-08-02CI: Refactor ci reproducer (#37088)kwryankrattiger2-2/+10
* CI: Refactor ci reproducer * Autostart container * Reproducer paths match CI paths * Generate start scripts for docker and reproducer * CI: Add interactive and gpg options to reproduce-build * Interactive will determine if the docker container persists after running reproduction. * GPG path/url allow downloading GPG keys needed for binary cache download validation. This is important for running reproducer for protected CI jobs. * Add exit_on_failure option to CI scripts * CI: Add runtime option for reproducer
2023-08-02feat: move -N/--namespace(s) to common args, allow in `buildcache list` (#36719)Wouter Deconinck2-14/+16
`spack buildcache list` did not have a way to display the namespace of packages in the buildcache. This PR adds that functionality. For consistency's sake, it moves the `-N/--namespace` arg definition to the `common/arguments.py` and modifies `find`, `solve`, `spec` to use the common definition. Previously, `find` was using `--namespace` (singular) to control whether to display the namespace (it doesn't restrict the search to that namespace). The other commands were using `--namespaces` (plural). For backwards compatibility and for consistency with `--deps`, `--tags`, etc, the plural `--namespaces` was chosen. The argument parser ensures that `find --namespace` will continue to behave as before. Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-07-31Add `spack checksum --verify`, fix `--add` (#38458)Alec Scott2-4/+6
* Add rewrite of spack checksum to include --verify and better add versions to package.py files * Fix formatting and remove unused import * Update checksum unit-tests to correctly test multiple versions and add to package * Remove references to latest in stage.py * Update bash-completion scripts to fix unit tests failures * Fix docs generation * Remove unused url_dict argument from methods * Reduce chance of redundant remote_versions work * Add print() before tty.die() to increase error readablity * Update version regular expression to allow for multi-line versions * Add a few unit tests to improve test coverage * Update command completion * Add type hints to added functions and fix a few py-lint suggestions * Add @no_type_check to prevent mypy from failing on pkg.versions * Add type hints to format.py and fix unit test * Black format lib/spack/spack/package_base.py * Attempt ignoring type errors * Add optional dict type hint and declare versions in PackageBase * Refactor util/format.py to allow for url_dict as an optional parameter * Directly reference PackageBase class instead of using TypeVar * Fix comment typo --------- Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
2023-07-26ci: Make signing requirement explicit (#38995)Scott Wittenburg1-0/+3
Instead of inferring whether to sign binaries, make it explicit, and fail rebuild jobs early if signing is required but cannot be accomplished.
2023-07-24ci: fix stack mirror urls (#38882)Scott Wittenburg2-1/+6
2023-07-22e4s oneapi ci: build with latest 2023.2 based image (#39048)eugeneswalker2-8/+8
2023-07-22Implement fish completion (#29549)百地 希留耶3-0/+3384
* commands: provide more information to Command * fish: Add script to generate fish completion * fish: auto prepend `spack` command to avoid duplication * fish: impove completion generation code readability * commands: replace match-case with if-else * fish: fix optspec variable name prefix * fish: fix return value in get_optspecs * fish: fix return value in get_optspecs * format: split long line and trim trailing space * bugfix: replace f-string with interpolation * fish: compete more specs and some fixes * fish: complete hash spec starts with / * fish: improve compatibility * style: trim trailing whitespace * commands: add fish to update args and update tests * commands: add fish completion file * style: merge imports * fish: source completion in setup-env * fish: caret only completes dependencies * fish: make sure we always get same order of output * fish: spack activate only show installed packages that have extensions * fish: update completion file * fish: make dict keys sorted * Blacken code * Fix bad merge * Undo style changes to setup-env.fish * Fix unit tests * Style fix * Compatible with fish_indent * Use list for stability of order * Sort one more place * Sort more things * Sorting unneeded * Unsort * Print difference * Style fix * Help messages need quotes * Arguments to -a must be quoted * Update types * Update types * Update types * Add type hints * Change order of positionals * Always expand help * Remove shared base class * Fix type hints * Remove platform-specific choices * First line of help only * Remove unused maps * Remove suppress * Remove debugging comments * Better quoting * Fish completions have no double dash * Remove test for deleted class * Fix grammar in header file * Use single quotes in most places * Better support for remainder nargs * No magic strings * * and + can also complete multiple * lower case, no period --------- Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2023-07-21e4s ci: add cray-sles ministack (#38744)eugeneswalker16-139/+548
* e4s ci: add cray-sles ministack * fix typo: variables, not env
2023-07-19modules: use curly braces to enclose value in Tcl modulefile (#38375)Xavier Delaruelle1-11/+11
Use curly braces instead of quotes to enclose value or text in Tcl modulefile. Within curly braces Tcl special characters like [, ] or $ are treated verbatim whereas they are evaluated within quotes. Curly braces is Tcl recommended way to enclose verbatim content [1]. Note: if curly braces charaters are used within content, they must be balanced. This point has been checked against current repository and no unbalanced curly braces has been spotted. Fixes #24243 [1] https://wiki.tcl-lang.org/page/Tcl+Minimal+Escaping+Style
2023-07-18Ensure the bootstrap store has a padding length of zero (#38975)Massimiliano Culpo1-0/+1
Without this PR, padded length was propagating from user configuration to bootstrap configuration, and was causing the issue reported in #38963
2023-07-18e4s cray: expand spec list (#38947)eugeneswalker1-0/+13
* e4s cray: expand spec list * unzip: require %gcc * remove datatrasnferkit
2023-07-18modules: add support for conflict in lua modulefile (#36701)Xavier Delaruelle1-0/+6
Add support for conflict directives in Lua modulefile like done for Tcl modulefile. Note that conflicts are correctly honored on Lmod and Environment Modules <4.2 only if mutually expressed on both modulefiles that conflict with each other. Migrate conflict code from Tcl-specific classes to the common part. Add tests for Lmod and split the conflict test case in two. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-07-14ci: remove aws-ahug (#38777)Harmen Stoppels4-530/+0
2023-07-13spack buildcache push: improve argparse (#38876)Harmen Stoppels1-2/+2
2023-07-13mirrors: distinguish between source/binary mirror; simplify schema (#34523)Harmen Stoppels1-3/+12
Allow the following formats: ```yaml mirrors: name: <url> ``` ```yaml mirrors: name: url: s3://xyz access_pair: [x, y] ``` ```yaml mirrors: name: fetch: http://xyz push: url: s3://xyz access_pair: [x, y] ``` And reserve two new properties to indicate the mirror type (e.g. mirror.spack.io is a source mirror, not a binary cache) ```yaml mirrors: spack-public: source: true binary: false url: https://mirror.spack.io ```
2023-07-12ci: populate caches in before script (#38762)Harmen Stoppels1-0/+2
* ci: run spack list in power ci Let's see if Spack itself is the bottleneck in CI... * rebuild curl in CI * more of the same please! * drop the profiler * undo rebuildme test in ci variant * add comment for posterity * enable profiling * trigger CI * See how it goes now that perf regressions are fixed on develop * try shorter poll intervals * Revert "try shorter poll intervals" This reverts commit d60c34ad3eceead0c13a5277cf8e783fd42b7458. * Remove spec.format call in Database._get_matching_spec_key * once more in ci please * undo irrelevant changes * run spack list in before script * test in ci * -: * Undo CI testing
2023-07-12containers: retain shallow git data (#37734)Jonathon Anderson1-1/+1
2023-07-12Fix build of CentOS stream docker image (#38824)Massimiliano Culpo1-4/+4
2023-07-12installer: Improve status reporting (#37903)Michael Kuhn1-1/+1
Refactor `TermTitle` into `InstallStatus` and use it to show progress information both in the terminal title as well as inline. This also turns on the terminal title status by default. The inline output will look like the following after this change: ``` ==> Installing m4-1.4.19-w2fxrpuz64zdq63woprqfxxzc3tzu7p3 [4/4] ```
2023-07-10find: add `--hashes` shortcut for piping to other commands (#38663)Todd Gamblin1-1/+1
People frequently ask us how to pipe `spack find` output to other commands, and we tell them to do things like this: ```console $ spack find --format "/{hash}" | spack uninstall -ay ``` Sometimes users don't know about hash references and come up with potentially ambiguous formulations like this: ```console spack find --format {name}@{version}%{compiler} | spack uninstall -ay ``` Since this is a common enough thing to want to do, and to make it more obvious how, this PR adds a `-H` / `--hashes` as a shortcut, so you can now just do: ```console spack find -H | spack uninstall -ay ```
2023-07-09openssl: prefer 3.x (#36729)Harmen Stoppels1-1/+1
* openssl: prefer 3.x This PR is not intended to be merged immediately, but it would be good to see what packages fail to build in CI so that we can get proper version constraints on openssl (before all packages update and support both openssl 1 and 3) * Disable assembly for 3.x %oneapi * cmake: depend on spack curl, to deal with curl - openssl compat * also make zlib external * remove overly strict & unsafe requirement on py-cryptographty patch version number * update openssl compat bounds in py-cryptography * smaller diff * Make libssh2 an autotools/cmake package * fix weird upperbound in libssh2 as there is not openssl v2 * libssh2: pc file lists plain -lssl -lcrypto w/o leading -L flag, confusing libgit2 parsing of pkg-config output * Actually fix the issue in libssh2: its pc file looks broken
2023-07-06Disable fortran in openblas for darwin ci, fix variant default value (#38752)Harmen Stoppels1-0/+2
2023-07-04remove another Python 3.7 requirement & preference, since it was deprecated ↵Harmen Stoppels3-26/+18
(#38710)
2023-07-04Temporarily disable aws-pcluster pipelines (#38708)Stephen Sachs1-48/+48
2023-07-04macos sip: apply on macos only, dont store LD_LIBRARY_PATH (#38687)Harmen Stoppels2-6/+3
2023-07-04Fix DYLD_* propagation to Python process from fish shell (#38615)Adam J. Stewart1-1/+8
2023-07-03Drop requirement of python@3.7 since it's deprecated (#38692)Harmen Stoppels1-2/+0
2023-06-29CI: Disable building VisIt in CI (#38642)kwryankrattiger1-3/+3
VisIt requires a deprecated version of Python (3.7) due to a VTK 8 dependency. Spack CI does not support building deprecated versions.
2023-06-28e4s ci: add hdf5-vol-daos (#37887)eugeneswalker2-5/+7
* e4s ci: add hdf5-vol-daos * hdf5-vol-daos: comment with failure notes
2023-06-27Windows: Add PowerShell env support (#37951)John W. Parent2-1/+5
PowerShell requires explicit shell and env support in Spack. This is due to the distinct differences in shell interactions between cmd and pwsh. Add a doskey in pwsh piping 'spack' commands to a powershell script similar to the sh function 'spack'. Add support for PowerShell-specific shell interactions from Spack (set/unset shell variables).
2023-06-27Kokkos Kernels: adding release 3.7.1 and 4.0.0 (#36655)Luc Berger2-2/+2
Co-authored-by: eugeneswalker <38933153+eugeneswalker@users.noreply.github.com>
2023-06-24Fix syntax to set default target in aws-pcluster pipelines (#38048)Stephen Sachs5-5/+5
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-06-16WarpX 23.06 (#38303)Axel Huebl2-6/+2
* WarpX 23.06 Update WarpX and related Python packages to the lastest releases. WarpX 23.06 introduces multi-dimension support in a single package, which will ease deployment in E4S et al. that can ship now a single, full-feature module/package that is NOT incompatible with itself anymore. * e4s ci stacks: multiple specs for each dim variant no longer required * [@spackbot] updating style on behalf of ax3l * WarpX: Update CMake CLI and Test/Check * Add Missing `build-directory` * [@spackbot] updating style on behalf of ax3l * Remove `build_directory` again --------- Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com> Co-authored-by: ax3l <ax3l@users.noreply.github.com>
2023-06-14modules: use depends-on to autoload module with Lmod on Tcl (#38347)Xavier Delaruelle1-1/+9
Update Tcl modulefile template to use the `depends-on` command to autoload modules if Lmod is the current module tool. Autoloading modules with `module load` command in Tcl modulefile does not work well for Lmod at some extend. An attempt to unload then load designated module is performed each time such command is encountered. It may lead to a load storm that may not end correctly with large number of module dependencies. `depends-on` command should be used for Lmod instead of `module load`, as it checks if module is already loaded, and does not attempt to reload this module. Lua modulefile template already uses `depends_on` command to autoload dependencies. Thus it is already considered that to use Lmod with Spack, it must support `depends_on` command (version 7.6+). Environment Modules copes well with `module load` command to autoload dependencies (version 3.2+). `depends-on` command is supported starting version 5.1 (as an alias of `prereq-all` command) which was relased last year. This change introduces a test to determine if current module tool that evaluates modulefile is Lmod. If so, autoload dependencies are defined with `depends-on` command. Otherwise `module load` command is used. Test is based on `LMOD_VERSION_MAJOR` environment variable, which is set by Lmod starting version 5.1. Fixes #36764
2023-06-13modules: append trailing delimiter to MANPATH when set (#36678)Xavier Delaruelle2-0/+8
Update modulefile templates to append a trailing delimiter to MANPATH environment variable, if the modulefile sets it. With a trailing delimiter at ends of MANPATH's value, man will search the system man pages after searching the specific paths set. Using append-path/append_path to add this element, the module tool ensures it is appended only once. When modulefile is unloaded, the number of append attempt is decreased, thus the trailing delimiter is removed only if this number equals 0. Disclaimer: no path element should be appended to MANPATH by generated modulefiles. It should always be prepended to ensure this variable's value ends with the trailing delimiter. Fixes #11355.
2023-06-12buildcache: remove deprecated api (#37246)Harmen Stoppels1-6/+6
The API was deprecated in v0.20 and is slated for removal in v0.21
2023-06-09install status arg: update to default true and add inverse (#38252)Greg Becker1-2/+2
* install status arg: update to default true and add inverse * update bash completion * fix tests broken by output change
2023-06-09containerize: use an ENTRYPOINT script (#37769)Jonathon Anderson1-3/+9
2023-06-09Hotfix to avoid building compilers from sources in pcluster pipelines (#38256)Massimiliano Culpo5-5/+20
The pcluster image has am internal buildcache without an index. Also, we need to force reuse to avoid rebuilding GCC, since the default is to only reuse dependencies - and that is subject to changes in the GCC recipe.
2023-06-08e4s oneapi ci: use official intel oneapi-derived runner image (#38176)eugeneswalker2-156/+131
* e4s oneapi ci: use official intel oneapi-derived runner image * update oneapi image * tau builds ok, but only with libdrm - comment out for now, follow up with pr later
2023-06-06E4S Cray CI Stack (#37837)eugeneswalker6-1/+502
* e4s cray ci stack * e4s ci: add cray * add zen4 tag * WIP: new defintions just for cray * updates * remove ci signing job overrride, not necessary * echo $PATH and show modules loaded * add mirror * add external def for cray-libsci * comment out quantum-espresso * use /etc/protected-runner as key path * cray ci stack: do not remove tags: [spack, public] * make cray stack composable * generate job should run on public tagged runner, override default config:install_tree:root * CI: Use relative path in default script * CI: Use relative includes paths for shell runners * Use concrete_env_dir for relpath * ml-darwin-aarch64-mps: jax has bazel codesign issue --------- Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com> Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2023-06-06containers: don't install epel-release on Fedora (#37766)Jonathon Anderson7-36/+36
2023-06-03llvm: optionally build compiler-rt, libcxx and libunwind as runtimes (#32476)Sergey Kosukhin1-1/+1
* llvm: replace +omp_as_runtime with omp=runtime * llvm: fetch 'libomp-libflags-as-list.patch' from upstream repo * llvm: replace 'llvm14-hwloc-ompd.patch' with the official fix from upstream repo * llvm: fix-up for the black reformatting * llvm: fetch 'constexpr_longdouble.patch' from upstream repo * llvm: optionally build libcxx as a runtime * llvm: fetch 'llvm5-sanitizer-ustat.patch' from upstream repo * llvm: update 'sanitizer-ipc_perm_mode.patch' * llvm: refactor compiler conflicts when libcxx=project * llvm: fetch 'llvm_python_path.patch' from upstream repo * llvm: update comments and condition for 'xray_buffer_queue-cstddef.patch' * llvm: optionally build compiler-rt as a runtime * llvm: fetch 'lldb_external_ncurses-10.patch' from upstream repo * llvm: fetch 'llvm_py37.patch' from upstream repo * llvm: rename variant 'internal_unwind' to 'libunwind' * llvm: optionally build libunwind as a runtime * llvm: extend the list of maintainers * llvm: allow for explicit '~clang~flang~libomptarget~lldb~omp_debug~z3' * llvm: fetch 'llvm5-lld-ELF-Symbols.patch' from FreeBSD port repo * llvm: fetch most of 'missing-includes.patch' from upstream repo and reuse 'llvm-gcc11.patch' * llvm: regroup patches for missing include directives and drop compiler constraints for them * llvm: fetch 'llvm-gcc11.patch' from upstream repo * llvm: fetch 'no_cyclades.patch' from upstream repo * llvm: update comments and condition for 'no_cyclades9.patch' * llvm: rename variant 'omp' to 'openmp' * llvm: constrain and rename variant 'omp_tsan' to 'libomp_tsan' * llvm: rename variant 'omp_debug' to 'libomptarget_debug' * llvm: do not apply same patch twice * llvm: constrain and document the '*-thread.patch' patches * llvm: document the '~lld+libomptarget' conflict * llvm: update comments for the 'D133513.diff' patch
2023-05-30Bugfix: cray manifest parsing regression (#37909)Tiziano Müller1-1/+1
fa7719a changed syntax for specifying exact versions, which are required for some compiler specs (including those read as part of parsing a Cray manifest). This fixes that and also makes a couple other improvements to manifest parsing. * Instantiate compiler specs with exact versions (fixes #37893) * fix slingshot network detection (CPE 22.10+ has libcxi.so in /usr/lib64) * "spack external find": add arg to ignore default dir for cray manifests
2023-05-25update ci ml darwin keypath (#37927)eugeneswalker1-3/+3
2023-05-25Add macOS ML CI stacks (#36586)Adam J. Stewart6-172/+223
* Add macOS ML CI stacks * torchmeta is no longer maintained and requires ancient PyTorch * Add MXNet * update darwin aarch64 stacks * add darwin-aarch64 scoped config.yaml * remove unnecessary cleanup job * fix specifications * fix labels * fix labels * fix indent on tags specification * no tags for trigger jobs * try overriding tags in stack spack.yaml * do not use CI_STACK_CONFIG_SCOPES * incorporate config:install_tree:root: overrides and compiler defs * copy relevant ci-scoped config settings directly into stack spack.yaml * remove build-job-remove * spack ci generate: add debug flag * include cdash config directly in stack spack.yaml * customize build-job script section to avoid absolute paths * add any-job specification * tags: use aarch64-macos instead of aarch64 * generate tags: use aarch64-macos instead of aarch64 * do not add morepadding * use shared mirror; comment out known failures * remove any-job * nproc || true * comment out specs failing due to bazel from cache codesign issue --------- Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
2023-05-24Pcluster local buildcache (#37852)Stephen Sachs5-38/+23
* [pcluster pipeline] Use local buildcache instead of upstream spack Spack currently does not relocate compiler references from upstream spack installations. When using a buildcache we don't need an upstream spack. * gcc needs to be installed via postinstall to get correct deps * quantum-espresso@gcc@12.3.0 returns ICE on neoverse_{n,v}1 * Force gitlab to pull the new container * Revert "Force gitlab to pull the new container" This reverts commit 3af5f4cd88245138992deb2a46c17e6f85858d68. Seems the gitlab version does not yet support "pull_policy" in .gitlab-ci.yml * Gitlab keeps picking up wrong container. Renaming * Update containers once more after failed build
2023-05-23e4s ci: add dealii (#32484)eugeneswalker1-0/+1
2023-05-17Add aws-plcuster[-aarch64] stacks (#37627)Stephen Sachs9-0/+394
Add aws-plcuster[-aarch64] stacks. These stacks build packages defined in https://github.com/spack/spack-configs/tree/main/AWS/parallelcluster They use a custom container from https://github.com/spack/gitlab-runners which includes necessary ParallelCluster software to link and build as well as an upstream spack installation with current GCC and dependencies. Intel and ARM software is installed and used during the build stage but removed from the buildcache before the signing stage. Files `configs/linux/{arch}/ci.yaml` select the necessary providers in order to build for specific architectures (icelake, skylake, neoverse_{n,v}1).
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2024-11-05 06:30:33 +0000