summaryrefslogtreecommitdiff
path: root/share
AgeCommit message (Collapse)AuthorFilesLines
2023-09-11gitlab ci: Use image with awscli for rebuild-index job (#39636)Scott Wittenburg5-9/+10
Instead of pointing to the image on DockerHub, which rate limits us and causes pipeline failures durying busy times, use the version at ghcr. And we might as well use the ghcr version everywhere else too.
2023-09-08bugfix: only complete aliases for potential aliases (#39887)Todd Gamblin3-4/+11
Smart alias completion introduced in #39499 wasn't as smart as it needed to be, and would complete any invalid command prefix and some env names with alias names. - [x] don't complete aliases if there are no potential completions e.g., don't convert `spack isnotacommand` -> `spack concretize` - [x] don't complete with an aliases if we're not looking at a top-level subcommand.
2023-09-08Speed-up `spack external find` execution (#39843)Massimiliano Culpo2-2/+4
* Perform external spec detection with multiple workers The logic to perform external spec detection has been refactored into classes. These classes use the GoF "template" pattern to account for the small differences between searching for "executables" and for "libraries", while unifying the larger part of the algorithm. A ProcessPoolExecutor is used to parallelize the work. * Speed-up external find by tagging detectable packages automatically Querying packages by tag is much faster than inspecting the repository, since tags are cached. This commit adds a "detectable" tag to every package that implements the detection protocol, and external detection uses it to search for packages. * Pass package names instead of package classes to workers The slowest part of the search is importing the Python modules associated with candidate packages. The import is done serially before we distribute the work to the pool of executors. This commit pushes the import of the Python module to the job performed by the workers, and passes just the name of the packages to the executors. In this way imports can be done in parallel. * Rework unit-tests for Windows Some unit tests were doing a full e2e run of a command just to check a input handling. Make the test more focused by just stressing a specific function. Mark as xfailed 2 tests on Windows, that will be fixed by a PR in the queue. The tests are failing because we monkeypatch internals in the parent process, but the monkeypatching is not done in the "spawned" child process.
2023-09-07CI Timing Statistics (#38598)kwryankrattiger2-1/+43
* Write timing information for installs from cache * CI: aggregate and upload install_times.json to artifacts * CI: Don't change root directory for artifact generation * Flat event based timer variation Event based timer allows for easily starting and stopping timers without wiping sub-timer data. It also requires less branching logic when tracking time. The json output is non-hierarchical in this version and hierarchy is less rigidly enforced between starting and stopping. * Add and write timers for top level install * Update completion * remove unused subtimer api * Fix unit tests * Suppress timing summary option * Save timers summaries to user_data artifacts * Remove completion from fish * Move spack python to script section * Write timer correctly for non-cache installs * Re-add hash to timer file * Fish completion updates * Fix null timer yield value * fix type hints * Remove timer-summary-file option * Add "." in front of non-package timer name --------- Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com> Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-09-06gitlab: set cache and config dir in top-level file (#39829)Harmen Stoppels1-2/+4
2023-09-05Set backtrace in ci (#39737)Harmen Stoppels1-0/+1
2023-09-01Windows: fix pwsh env activate/deactivate; load/unload (#39118)John W. Parent2-6/+12
These commands are currently broken on powershell (Windows) due to improper use of the InvokeCommand commandlet and a lack of direct support for the `--pwsh` argument in `spack load`, `spack unload`, and `spack env deactivate`.
2023-08-31ASP-based solver: add hidden mode to ignore versions that are moving ↵Harmen Stoppels1-0/+2
targets, use that in CI (#39611) Setting the undocumented variable SPACK_CONCRETIZER_REQUIRE_CHECKSUM now causes the solver to avoid accounting for versions that are not checksummed. This feature is used in CI to avoid spurious concretization against e.g. develop branches.
2023-08-30completion: make bash completion work in zshTodd Gamblin2-4/+32
`compgen -W` does not behave the same way in zsh as it does in bash; it seems not to actually generate the completions we want. - [x] add a zsh equivalent and `_compgen_w` to abstract it away - [x] use `_compgen_w` instead of `compgen -W`
2023-08-30completion: add alias handlingTodd Gamblin3-0/+100
Bash completion is now smarter about handling aliases. In particular, if all completions for some input command are aliased to the same thing, we'll just complete with that thing. If you've already *typed* the full alias for a command, we'll complete the alias. So, for example, here there's more than one real command involved, so all aliases are shown: ```console $ spack con concretise concretize config containerise containerize ``` Here, there are two possibilities: `concretise` and `concretize`, but both map to `concretize` so we just complete that: ```console $ spack conc concretize ``` And here, the user has already typed `concretis`, so we just go with it as there is only one option: ```console spack concretis concretise ```
2023-08-30Add `concretise` and `containerise` aliases for our UK usersTodd Gamblin2-1/+39
From a user: > Aargh. > ``` > ==> Error: concretise is not a recognized Spack command or extension command; check with `spack commands`. > ``` To make things easier for our friends in the UK, this adds `concretise` and `containerise` aliases for the `spack concretize` and `spack containerize` commands. - [x] add aliases - [x] update completions
2023-08-29Revert "Add style tool to fix fish file formatting (#39155)" (#39680)Harmen Stoppels2-397/+397
This reverts commit 70c71e8f935b3c5152036a993295df13c5fe69ca.
2023-08-28Add style tool to fix fish file formatting (#39155)Adam J. Stewart2-397/+397
2023-08-28Patch VTK to enable python 3.8 in VTK 8.2 (#38735)kwryankrattiger2-50/+50
* VTK: Add patch for python 3.8 support * CI: Re-enable VisIt in CI * Configure spec matrix for stack with VisIt * Add pugixml dep for 8.2.0 * Make VTK and ParaView consistent on proj dep * OpenMPI 3: provides MP support by default * Add details on proj dep in ParaView * Add python 3.8 to test mock repo * Patches to get VisIt VTK interface * CI: Disable VisIt with GUI in DAV
2023-08-28Remove mxnet from darwin aarch64 since theres no supported release (#39644)Harmen Stoppels1-2/+2
2023-08-23Add --fail-fast option for generating build caches (#38496)Dom Heinzeller2-4/+8
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-08-16amrex+sycl: restructure constraint on %oneapi (#39451)eugeneswalker1-1/+1
2023-08-15Change semantic for providersMassimiliano Culpo1-1/+1
If a possible provider is not used to satisfy a vdep, then it's not a provider of that vdep.
2023-08-15ecp-data-viz-sdk: fix building with new encodingMassimiliano Culpo1-3/+4
2023-08-15We cannot require "mpich" as an mpi provider and ask for openmpiMassimiliano Culpo1-2/+2
2023-08-14Fix containerize view symlink issue (#39419)Harmen Stoppels2-6/+11
2023-08-11e4s cray sles ci: expand spec list (#39081)eugeneswalker1-0/+12
* e4s cray sles ci: expand spec list * flux-core: disable until python+gary-sles issue is resolved
2023-08-11kokkos: rename 'std' variant to 'cxxstd'. (#39319)Cristian Di Pietrantonio1-4/+4
Co-authored-by: Cristian Di Pietrantonio <cdipietrantonio@pawsey.org.au>
2023-08-11Revert "package import: remove magic import line (#39183)" (#39380)Todd Gamblin1-2/+0
This reverts commit 8e7c53a8ba32e0793e36e66b5fdac8f4f61e644d.
2023-08-09AMReX: 23.06+ Multi-Dim Support (#37695)Axel Huebl2-3/+4
* AMReX: 23.06+ Multi-Dim Support This updated the Spack package to allow to install AMReX, modules of AMReX in E4S deployments and dependent packages with support for multiple dimensions. Due to an upstream change in AMReX, we do not longer need to ship three, binary incompatible package variants. * [E4S] oneAPI AMReX < 23.06 Variant Work-around the auto-concretization to the multi-dim of `dimensions`, which only in 23.06+ became a multi-variant. * e4s cray rhel ci: temporarily disable amrex build until spurious ci failure can be resolved --------- Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
2023-08-09package import: remove magic import line (#39183)Harmen Stoppels1-0/+2
2023-08-09Fix suffix of tab completion scripts (#39154)Adam J. Stewart4-4/+4
2023-08-09containerize: ensure bootstrap images contain all system dependencies (#36818)Jonathon Anderson6-1/+17
This also makes `spack bootstrap status` exit 1 if some dependency is missing
2023-08-08e4s ci: add libpressio (#34222)eugeneswalker1-0/+5
* e4s ci: add libpressio * libpressio ~cuda, +cuda: add +mgard
2023-08-08ML CI: get more packages working on macOS (#39199)Adam J. Stewart1-24/+27
2023-08-07CI: remove redundant sections (#38514)kwryankrattiger9-716/+70
Refactor gitlab ci configs so that mac and cray jobs can reuse as much higher level configuration as possible. * CI: remove redundant sections * CI: Include base linux CI configs in cray stacks Relocation and runner mapping is consistent between cray and linux runners. * Export user cache path in before script * CI: add GPG root for mac runners * Disable user configs Metal runners share a ~ directory * Disable user config and add configs in activate env
2023-08-02CI: Refactor ci reproducer (#37088)kwryankrattiger2-2/+10
* CI: Refactor ci reproducer * Autostart container * Reproducer paths match CI paths * Generate start scripts for docker and reproducer * CI: Add interactive and gpg options to reproduce-build * Interactive will determine if the docker container persists after running reproduction. * GPG path/url allow downloading GPG keys needed for binary cache download validation. This is important for running reproducer for protected CI jobs. * Add exit_on_failure option to CI scripts * CI: Add runtime option for reproducer
2023-08-02feat: move -N/--namespace(s) to common args, allow in `buildcache list` (#36719)Wouter Deconinck2-14/+16
`spack buildcache list` did not have a way to display the namespace of packages in the buildcache. This PR adds that functionality. For consistency's sake, it moves the `-N/--namespace` arg definition to the `common/arguments.py` and modifies `find`, `solve`, `spec` to use the common definition. Previously, `find` was using `--namespace` (singular) to control whether to display the namespace (it doesn't restrict the search to that namespace). The other commands were using `--namespaces` (plural). For backwards compatibility and for consistency with `--deps`, `--tags`, etc, the plural `--namespaces` was chosen. The argument parser ensures that `find --namespace` will continue to behave as before. Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-07-31Add `spack checksum --verify`, fix `--add` (#38458)Alec Scott2-4/+6
* Add rewrite of spack checksum to include --verify and better add versions to package.py files * Fix formatting and remove unused import * Update checksum unit-tests to correctly test multiple versions and add to package * Remove references to latest in stage.py * Update bash-completion scripts to fix unit tests failures * Fix docs generation * Remove unused url_dict argument from methods * Reduce chance of redundant remote_versions work * Add print() before tty.die() to increase error readablity * Update version regular expression to allow for multi-line versions * Add a few unit tests to improve test coverage * Update command completion * Add type hints to added functions and fix a few py-lint suggestions * Add @no_type_check to prevent mypy from failing on pkg.versions * Add type hints to format.py and fix unit test * Black format lib/spack/spack/package_base.py * Attempt ignoring type errors * Add optional dict type hint and declare versions in PackageBase * Refactor util/format.py to allow for url_dict as an optional parameter * Directly reference PackageBase class instead of using TypeVar * Fix comment typo --------- Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
2023-07-26ci: Make signing requirement explicit (#38995)Scott Wittenburg1-0/+3
Instead of inferring whether to sign binaries, make it explicit, and fail rebuild jobs early if signing is required but cannot be accomplished.
2023-07-24ci: fix stack mirror urls (#38882)Scott Wittenburg2-1/+6
2023-07-22e4s oneapi ci: build with latest 2023.2 based image (#39048)eugeneswalker2-8/+8
2023-07-22Implement fish completion (#29549)百地 希留耶3-0/+3384
* commands: provide more information to Command * fish: Add script to generate fish completion * fish: auto prepend `spack` command to avoid duplication * fish: impove completion generation code readability * commands: replace match-case with if-else * fish: fix optspec variable name prefix * fish: fix return value in get_optspecs * fish: fix return value in get_optspecs * format: split long line and trim trailing space * bugfix: replace f-string with interpolation * fish: compete more specs and some fixes * fish: complete hash spec starts with / * fish: improve compatibility * style: trim trailing whitespace * commands: add fish to update args and update tests * commands: add fish completion file * style: merge imports * fish: source completion in setup-env * fish: caret only completes dependencies * fish: make sure we always get same order of output * fish: spack activate only show installed packages that have extensions * fish: update completion file * fish: make dict keys sorted * Blacken code * Fix bad merge * Undo style changes to setup-env.fish * Fix unit tests * Style fix * Compatible with fish_indent * Use list for stability of order * Sort one more place * Sort more things * Sorting unneeded * Unsort * Print difference * Style fix * Help messages need quotes * Arguments to -a must be quoted * Update types * Update types * Update types * Add type hints * Change order of positionals * Always expand help * Remove shared base class * Fix type hints * Remove platform-specific choices * First line of help only * Remove unused maps * Remove suppress * Remove debugging comments * Better quoting * Fish completions have no double dash * Remove test for deleted class * Fix grammar in header file * Use single quotes in most places * Better support for remainder nargs * No magic strings * * and + can also complete multiple * lower case, no period --------- Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2023-07-21e4s ci: add cray-sles ministack (#38744)eugeneswalker16-139/+548
* e4s ci: add cray-sles ministack * fix typo: variables, not env
2023-07-19modules: use curly braces to enclose value in Tcl modulefile (#38375)Xavier Delaruelle1-11/+11
Use curly braces instead of quotes to enclose value or text in Tcl modulefile. Within curly braces Tcl special characters like [, ] or $ are treated verbatim whereas they are evaluated within quotes. Curly braces is Tcl recommended way to enclose verbatim content [1]. Note: if curly braces charaters are used within content, they must be balanced. This point has been checked against current repository and no unbalanced curly braces has been spotted. Fixes #24243 [1] https://wiki.tcl-lang.org/page/Tcl+Minimal+Escaping+Style
2023-07-18Ensure the bootstrap store has a padding length of zero (#38975)Massimiliano Culpo1-0/+1
Without this PR, padded length was propagating from user configuration to bootstrap configuration, and was causing the issue reported in #38963
2023-07-18e4s cray: expand spec list (#38947)eugeneswalker1-0/+13
* e4s cray: expand spec list * unzip: require %gcc * remove datatrasnferkit
2023-07-18modules: add support for conflict in lua modulefile (#36701)Xavier Delaruelle1-0/+6
Add support for conflict directives in Lua modulefile like done for Tcl modulefile. Note that conflicts are correctly honored on Lmod and Environment Modules <4.2 only if mutually expressed on both modulefiles that conflict with each other. Migrate conflict code from Tcl-specific classes to the common part. Add tests for Lmod and split the conflict test case in two. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-07-14ci: remove aws-ahug (#38777)Harmen Stoppels4-530/+0
2023-07-13spack buildcache push: improve argparse (#38876)Harmen Stoppels1-2/+2
2023-07-13mirrors: distinguish between source/binary mirror; simplify schema (#34523)Harmen Stoppels1-3/+12
Allow the following formats: ```yaml mirrors: name: <url> ``` ```yaml mirrors: name: url: s3://xyz access_pair: [x, y] ``` ```yaml mirrors: name: fetch: http://xyz push: url: s3://xyz access_pair: [x, y] ``` And reserve two new properties to indicate the mirror type (e.g. mirror.spack.io is a source mirror, not a binary cache) ```yaml mirrors: spack-public: source: true binary: false url: https://mirror.spack.io ```
2023-07-12ci: populate caches in before script (#38762)Harmen Stoppels1-0/+2
* ci: run spack list in power ci Let's see if Spack itself is the bottleneck in CI... * rebuild curl in CI * more of the same please! * drop the profiler * undo rebuildme test in ci variant * add comment for posterity * enable profiling * trigger CI * See how it goes now that perf regressions are fixed on develop * try shorter poll intervals * Revert "try shorter poll intervals" This reverts commit d60c34ad3eceead0c13a5277cf8e783fd42b7458. * Remove spec.format call in Database._get_matching_spec_key * once more in ci please * undo irrelevant changes * run spack list in before script * test in ci * -: * Undo CI testing
2023-07-12containers: retain shallow git data (#37734)Jonathon Anderson1-1/+1
2023-07-12Fix build of CentOS stream docker image (#38824)Massimiliano Culpo1-4/+4
2023-07-12installer: Improve status reporting (#37903)Michael Kuhn1-1/+1
Refactor `TermTitle` into `InstallStatus` and use it to show progress information both in the terminal title as well as inline. This also turns on the terminal title status by default. The inline output will look like the following after this change: ``` ==> Installing m4-1.4.19-w2fxrpuz64zdq63woprqfxxzc3tzu7p3 [4/4] ```