Bootstrap clingo from binaries (#22720)

* Bootstrap clingo from binaries

* Move information on clingo binaries to a JSON file

* Add support to bootstrap on Cray

Bootstrapping on Cray requires, at the moment, to
swap the platform when looking for binaries - due
to #22800.

* Add SHA256 verification for bootstrapped software

Use sha256 verification for binaries necessary to bootstrap
the concretizer and gpg for signature verification

* patchelf: use Spec._old_concretize() to bootstrap

As noted in #24450 we may happen to need the
concretizer when bootstrapping clingo. In that case
only the old concretizer is available.

* Add a schema for bootstrapping methods

Two fields have been added to bootstrap.yaml:
  "sources" which lists the methods available for
       bootstrapping software
  "trusted" which records if a source is trusted or not

A subcommand has been added to "spack bootstrap" to list
the sources currently available.

* Methods used for bootstrapping are configurable from bootstrap:sources

The function that tries to ensure a given Python module
is importable now tries bootstrapping methods in the same
order as they are defined in `bootstrap.yaml`

* Permit to trust/untrust bootstrapping methods

* Add binary tests for MacOS, Ubuntu

* Add documentation

* Add a note on bash

2 files changed, 53 insertions, 6 deletions

diff --git a/.github/workflows/bootstrap.yml b/.github/workflows/bootstrap.yml
index 70fe9bf58a..eebdd9aa53 100644
--- a/.github/workflows/bootstrap.yml
+++ b/.github/workflows/bootstrap.yml
@@ -18,7 +18,7 @@ on:
 
 jobs:
 
-  fedora:
+  fedora-sources:
     runs-on: ubuntu-latest
     container: "fedora:latest"
     steps:
@@ -40,11 +40,12 @@ jobs:
         shell: runuser -u spack-test -- bash {0}
         run: |
           source share/spack/setup-env.sh
+          spack bootstrap untrust github-actions
           spack external find cmake bison
           spack -d solve zlib
           tree ~/.spack/bootstrap/store/
 
-  ubuntu:
+  ubuntu-sources:
     runs-on: ubuntu-latest
     container: "ubuntu:latest"
     steps:
@@ -69,11 +70,12 @@ jobs:
         shell: runuser -u spack-test -- bash {0}
         run: |
           source share/spack/setup-env.sh
+          spack bootstrap untrust github-actions
           spack external find cmake bison
           spack -d solve zlib
           tree ~/.spack/bootstrap/store/
 
-  opensuse:
+  opensuse-sources:
     runs-on: ubuntu-latest
     container: "opensuse/tumbleweed:latest"
     steps:
@@ -93,11 +95,12 @@ jobs:
       - name: Bootstrap clingo
         run: |
           source share/spack/setup-env.sh
+          spack bootstrap untrust github-actions
           spack external find cmake bison
           spack -d solve zlib
           tree ~/.spack/bootstrap/store/
 
-  macos:
+  macos-sources:
     runs-on: macos-latest
     steps:
       - name: Install dependencies
@@ -108,6 +111,50 @@ jobs:
         run: |
           source share/spack/setup-env.sh
           export PATH=/usr/local/opt/bison@2.7/bin:$PATH
+          spack bootstrap untrust github-actions
           spack external find --not-buildable cmake bison
           spack -d solve zlib
           tree ~/.spack/bootstrap/store/
+
+  macos-clingo-binaries:
+    runs-on: macos-latest
+    strategy:
+      matrix:
+        python-version: ['3.5', '3.6', '3.7', '3.8', '3.9']
+    steps:
+      - name: Install dependencies
+        run: |
+          brew install tree
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Bootstrap clingo
+        run: |
+          source share/spack/setup-env.sh
+          spack bootstrap untrust spack-install
+          spack -d solve zlib
+          tree ~/.spack/bootstrap/store/
+
+
+  ubuntu-clingo-binaries:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ['2.7', '3.5', '3.6', '3.7', '3.8', '3.9']
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Setup repo and non-root user
+        run: |
+          git --version
+          git fetch --unshallow
+          . .github/workflows/setup_git.sh
+      - name: Bootstrap clingo
+        run: |
+          source share/spack/setup-env.sh
+          spack bootstrap untrust spack-install
+          spack -d solve zlib
+          tree ~/.spack/bootstrap/store/
diff --git a/.github/workflows/unit_tests.yaml b/.github/workflows/unit_tests.yaml
index 32b1baf39a..b45249f371 100644
--- a/.github/workflows/unit_tests.yaml
+++ b/.github/workflows/unit_tests.yaml
@@ -159,13 +159,13 @@ jobs:
           mkdir -p ${KCOV_ROOT}/build
           cd ${KCOV_ROOT}/build && cmake -Wno-dev ${KCOV_ROOT}/kcov-${KCOV_VERSION} && cd -
           make -C ${KCOV_ROOT}/build && sudo  make -C ${KCOV_ROOT}/build install
-    - name: Bootstrap clingo from sources
+    - name: Bootstrap clingo
       if: ${{ matrix.concretizer == 'clingo' }}
       env:
           SPACK_PYTHON: python
       run: |
           . share/spack/setup-env.sh
-          spack external find --not-buildable cmake bison
+          spack bootstrap untrust spack-install
           spack -v solve zlib
     - name: Run unit tests (full suite with coverage)
       if: ${{ needs.changes.outputs.with_coverage == 'true' }}