From 5b0507cc6536f8b243d398f4e1b78b5eee298753 Mon Sep 17 00:00:00 2001 From: Scott Wittenburg Date: Tue, 16 Feb 2021 18:21:18 -0700 Subject: Pipelines: Temporary buildcache storage (#21474) Before this change, in pipeline environments where runners do not have access to persistent shared file-system storage, the only way to pass buildcaches to dependents in later stages was by using the "enable-artifacts-buildcache" flag in the gitlab-ci section of the spack.yaml. This change supports a second mechanism, named "temporary-storage-url-prefix", which can be provided instead of the "enable-artifacts-buildcache" feature, but the two cannot be used at the same time. If this prefix is provided (only "file://" and "s3://" urls are supported), the gitlab "CI_PIPELINE_ID" will be appended to it to create a url for a mirror where pipeline jobs will write buildcache entries for use by jobs in subsequent stages. If this prefix is provided, a cleanup job will be generated to run after all the rebuild jobs have finished that will delete the contents of the temporary mirror. To support this behavior a new mirror sub-command has been added: "spack mirror destroy" which can take either a mirror name or url. This change also fixes a bug in generation of "needs" list for each job. Each jobs "needs" list is supposed to only contain direct dependencies for scheduling purposes, unless "enable-artifacts-buildcache" is specified. Only in that case are the needs lists supposed to contain all transitive dependencies. This changes fixes a bug that caused the needs lists to always contain all transitive dependencies, regardless of whether or not "enable-artifacts-buildcache" was specified. --- share/spack/spack-completion.bash | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'share') diff --git a/share/spack/spack-completion.bash b/share/spack/spack-completion.bash index cf0e79cdac..62829d474a 100755 --- a/share/spack/spack-completion.bash +++ b/share/spack/spack-completion.bash @@ -1123,7 +1123,7 @@ _spack_mirror() { then SPACK_COMPREPLY="-h --help -n --no-checksum --deprecated" else - SPACK_COMPREPLY="create add remove rm set-url list" + SPACK_COMPREPLY="create destroy add remove rm set-url list" fi } @@ -1136,6 +1136,10 @@ _spack_mirror_create() { fi } +_spack_mirror_destroy() { + SPACK_COMPREPLY="-h --help -m --mirror-name --mirror-url" +} + _spack_mirror_add() { if $list_options then -- cgit v1.2.3-60-g2f50