summaryrefslogtreecommitdiff
path: root/.github/workflows
diff options
context:
space:
mode:
authorScott Wittenburg <scott.wittenburg@kitware.com>2022-11-07 13:31:14 -0700
committerGitHub <noreply@github.com>2022-11-07 21:31:14 +0100
commit28a77c2821d62a2800a63921f116fcf45a8f7342 (patch)
tree204a6842e65e37421597f1d4c3a59dcc543e9cd3 /.github/workflows
parent01a578851707281b8e1da736086b41e98bfbe5f4 (diff)
downloadspack-28a77c2821d62a2800a63921f116fcf45a8f7342.tar.gz
spack-28a77c2821d62a2800a63921f116fcf45a8f7342.tar.bz2
spack-28a77c2821d62a2800a63921f116fcf45a8f7342.tar.xz
spack-28a77c2821d62a2800a63921f116fcf45a8f7342.zip
binary_distribution: Speed up buildcache update-index (#32796)
This change uses the aws cli, if available, to retrieve spec files from the mirror to a local temp directory, then parallelizes the reading of those files from disk using multiprocessing.ThreadPool. If the aws cli is not available, then a ThreadPool is used to fetch and read the spec files from the mirror. Using aws cli results in ~16 times speed up to recreate the binary mirror index, while just parallelizing the fetching and reading results in ~3 speed up.
Diffstat (limited to '.github/workflows')
0 files changed, 0 insertions, 0 deletions