summaryrefslogtreecommitdiff
path: root/share/spack/gitlab/cloud_pipelines/configs
AgeCommit message (Collapse)AuthorFilesLines
2023-11-30developer tools stack try 2 (#40921)Tom Scogland1-1/+0
* developer tools stack try 2 This version is actually in use locally and has largely stabilized, at least on x86. Some packages are still a challenge on ppc64le, but maybe worth keeping this working as a set. * add packages, try to get container with newer gcc * remove reuse: true * try to get cmake to build on medium, 25 minutes is too long * add lsd package and add to dev tools stack * clean up fzf dependency and sorting * Update share/spack/gitlab/cloud_pipelines/stacks/developer_tools/spack.yaml * cuda: add 12.3.0 (#40827) * Switch to dashes * yet more underscores --------- Co-authored-by: Paul R. C. Kent <kentpr@ornl.gov>
2023-11-14Add papyrus to the list of broken tests (#40923)Massimiliano Culpo1-0/+1
* Disable papyrus in the neoverse v1 pipeline See https://gitlab.spack.io/spack/spack/-/jobs/8983875 The job is hanging on tests for 6 hrs. * Add papyrus to broken tests instead of removing it
2023-10-25ci: don't put compilers in config (#40700)Harmen Stoppels1-1/+1
* ci: don't register detectable compilers Cause they go out of sync... * remove intel compiler, it can be detected too * Do not run spack compiler find since compilers are registered in concretize job already * trilinos: work around +stokhos +cuda +superlu-dist bug due to EMPTY macro
2023-10-19gitlab ci: Rework how mirrors are configured (#39939)Scott Wittenburg5-9/+50
Improve how mirrors are used in gitlab ci, where we have until now thought of them as only a string. By configuring ci mirrors ahead of time using the proposed mirror templates, and by taking advantage of the expressiveness that spack now has for mirrors, this PR will allow us to easily switch the protocol/url we use for fetching binary dependencies. This change also deprecates some gitlab functionality and marks it for removal in Spack 0.23: - arguments to "spack ci generate": * --buildcache-destination * --copy-to - gitlab configuration options: * enable-artifacts-buildcache * temporary-storage-url-prefix
2023-10-19ci: remove incorrect compilers.yaml (#40610)Harmen Stoppels1-27/+0
2023-09-19ci: drop redundant packages.yaml (#40068)Harmen Stoppels1-14/+0
2023-09-14Add OIDC tokens to gitlab-ci jobs (#39813)Dan LaManna1-0/+15
* Add OIDC tokens to gitlab-ci jobs This should allow us to start issuing just-in-time generated credentials for CI jobs that need to modify binary mirrors. The "aud" claim of the token describes what the token is allowed to do. The claim is verified against a set of rules on the IAM role using signed information from GitLab. See spack-infrastructure for the claim verification logic. --------- Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
2023-09-12ci: Tag reindex job to match the image architecture (#39958)Scott Wittenburg1-1/+1
2023-09-12CI/Add superlu-dist to broken stand-alone tests due to hang (#38908)Tamara Dahlgren1-0/+1
* Add superlu-dist to broken stand-alone CI tests * Revert "disable superlu test (#38894)" This reverts commit e16397b5d865d402c749482d05bb2bdc09b96661.
2023-09-11gitlab ci: Use image with awscli for rebuild-index job (#39636)Scott Wittenburg1-0/+1
Instead of pointing to the image on DockerHub, which rate limits us and causes pipeline failures durying busy times, use the version at ghcr. And we might as well use the ghcr version everywhere else too.
2023-09-07CI Timing Statistics (#38598)kwryankrattiger1-1/+5
* Write timing information for installs from cache * CI: aggregate and upload install_times.json to artifacts * CI: Don't change root directory for artifact generation * Flat event based timer variation Event based timer allows for easily starting and stopping timers without wiping sub-timer data. It also requires less branching logic when tracking time. The json output is non-hierarchical in this version and hierarchy is less rigidly enforced between starting and stopping. * Add and write timers for top level install * Update completion * remove unused subtimer api * Fix unit tests * Suppress timing summary option * Save timers summaries to user_data artifacts * Remove completion from fish * Move spack python to script section * Write timer correctly for non-cache installs * Re-add hash to timer file * Fish completion updates * Fix null timer yield value * fix type hints * Remove timer-summary-file option * Add "." in front of non-package timer name --------- Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com> Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-08-28Patch VTK to enable python 3.8 in VTK 8.2 (#38735)kwryankrattiger1-11/+14
* VTK: Add patch for python 3.8 support * CI: Re-enable VisIt in CI * Configure spec matrix for stack with VisIt * Add pugixml dep for 8.2.0 * Make VTK and ParaView consistent on proj dep * OpenMPI 3: provides MP support by default * Add details on proj dep in ParaView * Add python 3.8 to test mock repo * Patches to get VisIt VTK interface * CI: Disable VisIt with GUI in DAV
2023-08-07CI: remove redundant sections (#38514)kwryankrattiger5-580/+22
Refactor gitlab ci configs so that mac and cray jobs can reuse as much higher level configuration as possible. * CI: remove redundant sections * CI: Include base linux CI configs in cray stacks Relocation and runner mapping is consistent between cray and linux runners. * Export user cache path in before script * CI: add GPG root for mac runners * Disable user configs Metal runners share a ~ directory * Disable user config and add configs in activate env
2023-07-21e4s ci: add cray-sles ministack (#38744)eugeneswalker12-13/+386
* e4s ci: add cray-sles ministack * fix typo: variables, not env
2023-07-14ci: remove aws-ahug (#38777)Harmen Stoppels1-8/+0
2023-07-12ci: populate caches in before script (#38762)Harmen Stoppels1-0/+2
* ci: run spack list in power ci Let's see if Spack itself is the bottleneck in CI... * rebuild curl in CI * more of the same please! * drop the profiler * undo rebuildme test in ci variant * add comment for posterity * enable profiling * trigger CI * See how it goes now that perf regressions are fixed on develop * try shorter poll intervals * Revert "try shorter poll intervals" This reverts commit d60c34ad3eceead0c13a5277cf8e783fd42b7458. * Remove spec.format call in Database._get_matching_spec_key * once more in ci please * undo irrelevant changes * run spack list in before script * test in ci * -: * Undo CI testing
2023-07-04remove another Python 3.7 requirement & preference, since it was deprecated ↵Harmen Stoppels1-22/+18
(#38710)
2023-06-06E4S Cray CI Stack (#37837)eugeneswalker3-0/+308
* e4s cray ci stack * e4s ci: add cray * add zen4 tag * WIP: new defintions just for cray * updates * remove ci signing job overrride, not necessary * echo $PATH and show modules loaded * add mirror * add external def for cray-libsci * comment out quantum-espresso * use /etc/protected-runner as key path * cray ci stack: do not remove tags: [spack, public] * make cray stack composable * generate job should run on public tagged runner, override default config:install_tree:root * CI: Use relative path in default script * CI: Use relative includes paths for shell runners * Use concrete_env_dir for relpath * ml-darwin-aarch64-mps: jax has bazel codesign issue --------- Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com> Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2023-05-25Add macOS ML CI stacks (#36586)Adam J. Stewart3-14/+30
* Add macOS ML CI stacks * torchmeta is no longer maintained and requires ancient PyTorch * Add MXNet * update darwin aarch64 stacks * add darwin-aarch64 scoped config.yaml * remove unnecessary cleanup job * fix specifications * fix labels * fix labels * fix indent on tags specification * no tags for trigger jobs * try overriding tags in stack spack.yaml * do not use CI_STACK_CONFIG_SCOPES * incorporate config:install_tree:root: overrides and compiler defs * copy relevant ci-scoped config settings directly into stack spack.yaml * remove build-job-remove * spack ci generate: add debug flag * include cdash config directly in stack spack.yaml * customize build-job script section to avoid absolute paths * add any-job specification * tags: use aarch64-macos instead of aarch64 * generate tags: use aarch64-macos instead of aarch64 * do not add morepadding * use shared mirror; comment out known failures * remove any-job * nproc || true * comment out specs failing due to bazel from cache codesign issue --------- Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
2023-05-17Add aws-plcuster[-aarch64] stacks (#37627)Stephen Sachs4-0/+36
Add aws-plcuster[-aarch64] stacks. These stacks build packages defined in https://github.com/spack/spack-configs/tree/main/AWS/parallelcluster They use a custom container from https://github.com/spack/gitlab-runners which includes necessary ParallelCluster software to link and build as well as an upstream spack installation with current GCC and dependencies. Intel and ARM software is installed and used during the build stage but removed from the buildcache before the signing stage. Files `configs/linux/{arch}/ci.yaml` select the necessary providers in order to build for specific architectures (icelake, skylake, neoverse_{n,v}1).
2023-05-12gitlab ci: release fixes and improvements (#37601)Scott Wittenburg1-0/+28
* gitlab ci: release fixes and improvements - use rules to reduce boilerplate in .gitlab-ci.yml - support copy-only pipeline jobs - make pipelines for release branches rebuild everything - make pipelines for protected tags copy-only * gitlab ci: remove url changes used in testing * gitlab ci: tag mirrors need public key Make sure that mirrors associated with release branches and tags contain the public key needed to verify the signed binaries. This also ensures that when stack-specific mirror contents are copied to the root, the root mirror has the public key as well. * review: be more specific about tags, curl flags * Make the check in ci.yaml consistent with the .gitlab-ci.yml --------- Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2023-05-05ci: stop downloading recent gmake (#37458)Harmen Stoppels2-8/+0
2023-04-14gitlab ci: Better tagging of "service" jobs (#36846)Scott Wittenburg1-0/+21
- Tag non-rebuild jobs to target a cheaper (and more highly available) subset of runners. - Add missing resource requests to these jobs as well.
2023-03-29ci: require x86_64_v3 everywhere (#36158)Harmen Stoppels2-9/+1
2023-03-13Set build_jobs dynamically in CI to avoid oversubscription (#35996)Harmen Stoppels4-2/+7
Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com> Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2023-03-10get --dev and drop set -x (#36010)Harmen Stoppels1-1/+1
2023-03-10curl flag is not universally supported (#36009)Harmen Stoppels3-3/+3
2023-03-10hotfix: fix double double quotes (#36005)Harmen Stoppels1-1/+1
2023-03-10CI boilerplate reduction (#34272)kwryankrattiger11-0/+439
* CI configuration boilerplate reduction and refactor Configuration: - New notation for list concatenation (prepend/append) - New notation for string concatenation (prepend/append) - Break out configuration files for: ci.yaml, cdash.yaml, view.yaml - Spack CI section refactored to improve self-consistency and composability - Scripts are now lists of lists and/or lists of strings - Job attributes are now listed under precedence ordered list that are composed/merged using Spack config merge rules. - "service-jobs" are identified explicitly rather than as a batch CI: - Consolidate common, platform, and architecture configurations for all CI stacks into composable configuration files - Make padding consistent across all stacks (256) - Merge all package -> runner mappings to be consistent across all stacks Unit Test: - Refactor CI module unit-tests for refactor configuration Docs: - Add docs for new notations in configuration.rst - Rewrite docs on CI pipelines to be consistent with refactored CI workflow * Script verbose environ, dev bootstrap * Port #35409
generated by cgit v1.2.3-60-g2f50 (git 2.42.0) at 2024-07-01 07:15:00 +0000