diff options
| author | Scott Wittenburg <scott.wittenburg@kitware.com> | 2022-09-01 15:29:44 -0600 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2022-09-01 15:29:44 -0600 |
| commit | 6239198d65d20d8dfbd15bd617168dd92523e0b7 (patch) | |
| tree | 6e2fe464a1000901639616c3e2e97bed7a4b0e53 /share | |
| parent | d9313cf561ff6866971bb7fb488e0ef1b6725b6e (diff) | |
| download | spack-6239198d65d20d8dfbd15bd617168dd92523e0b7.tar.gz spack-6239198d65d20d8dfbd15bd617168dd92523e0b7.tar.bz2 spack-6239198d65d20d8dfbd15bd617168dd92523e0b7.tar.xz spack-6239198d65d20d8dfbd15bd617168dd92523e0b7.zip | |
Fix cause of checksum failures in public binary mirror (#32407)
Move the copying of the buildcache to a root job that runs after all the child
pipelines have finished, so that the operation can be coordinated across all
child pipelines to remove the possibility of race conditions during potentially
simlutandous copies. This lets us ensure the .spec.json.sig and .spack files
for any spec in the root mirror always come from the same child pipeline
mirror (though which pipeline is arbitrary). It also allows us to avoid copying
of duplicates, which we now do.
Diffstat (limited to 'share')
| -rw-r--r-- | share/spack/gitlab/cloud_pipelines/.gitlab-ci.yml | 8 | ||||
| -rwxr-xr-x | share/spack/spack-completion.bash | 2 |
2 files changed, 6 insertions, 4 deletions
diff --git a/share/spack/gitlab/cloud_pipelines/.gitlab-ci.yml b/share/spack/gitlab/cloud_pipelines/.gitlab-ci.yml index 3bdbd2086a..a6174c20b3 100644 --- a/share/spack/gitlab/cloud_pipelines/.gitlab-ci.yml +++ b/share/spack/gitlab/cloud_pipelines/.gitlab-ci.yml @@ -12,7 +12,7 @@ default: - /^pr[\d]+_.*$/ - /^github\/pr[\d]+_.*$/ variables: - SPACK_BUILDCACHE_DESTINATION: "s3://spack-binaries-prs/${CI_COMMIT_REF_NAME}" + SPACK_BUILDCACHE_DESTINATION: "s3://spack-binaries-prs/${CI_COMMIT_REF_NAME}/${SPACK_CI_STACK_NAME}" SPACK_PIPELINE_TYPE: "spack_pull_request" SPACK_PRUNE_UNTOUCHED: "True" @@ -88,7 +88,7 @@ default: protected-publish: stage: publish - extends: [ ".protected-refs" ] + extends: [ ".protected" ] image: "ghcr.io/spack/python-aws-bash:0.0.1" tags: ["spack", "public", "medium", "aws", "x86_64"] variables: @@ -97,7 +97,9 @@ protected-publish: script: - . "./share/spack/setup-env.sh" - spack --version - - spack buildcache update-index --mirror-url "s3://spack-binaries/${CI_COMMIT_REF_NAME}" + - export COPY_SPECS_DIR=${CI_PROJECT_DIR}/jobs_scratch_dir/specs_to_copy + - spack buildcache sync --manifest-glob "${COPY_SPECS_DIR}/*.json" + - spack buildcache update-index --mirror-url ${SPACK_COPY_BUILDCACHE} ######################################## # TEMPLATE FOR ADDING ANOTHER PIPELINE diff --git a/share/spack/spack-completion.bash b/share/spack/spack-completion.bash index 58e94c036a..bde48d8e97 100755 --- a/share/spack/spack-completion.bash +++ b/share/spack/spack-completion.bash @@ -569,7 +569,7 @@ _spack_buildcache_copy() { } _spack_buildcache_sync() { - SPACK_COMPREPLY="-h --help --src-directory --src-mirror-name --src-mirror-url --dest-directory --dest-mirror-name --dest-mirror-url" + SPACK_COMPREPLY="-h --help --manifest-glob --src-directory --src-mirror-name --src-mirror-url --dest-directory --dest-mirror-name --dest-mirror-url" } _spack_buildcache_update_index() { |