cray platform: support cray Cluster and XC type machines (#12989)

Cray has two machine types. "XC" machines are the larger
machines more common in HPC, but "Cluster" machines are
also cropping up at some HPC sites. Cluster machines run
a slightly different form of the CrayPE programming environment,
and often come without default modules loaded. Cluster
machines also run different versions of some software, and run
a linux distro on the backend nodes instead of running Compute 
Node Linux (CNL).

Below are the changes made to support "Cluster" machines in
Spack. Some of these changes are semi-related general upkeep
of the cray platform.

* cray platform: detect properly after module purge

* cray platform: support machines running OSs other than CNL

Make Cray backend OS delegate to LinuxDistro when no cle_release file
favor backend over frontend OS when name clashes

* cray platform: target detection uses multiple strategies

This commit improves the robustness of target
detection on Cray by trying multiple strategies.

The first one that produces results wins. If
nothing is found only the generic family of the
frontend host is used as a target.

* cray-libsci: add package from NERSC

* build_env: unload cray-libsci module when not explicitly needed

cray-libsci is a package in Spack. The cray PrgEnv
modules load it implicitly when we set up the compiler.
We now unload it after setting up the compiler and
only reload it when requested via external package.

* util/module_cmd: more robust module parsing

Cray modules have documentation inside the module
that is visible to the `module show` command.
Spack module parsing is now robust to documentation 
inside modules.

* cce compiler: uses clang flags for versions >= 9.0

* build_env: push CRAY_LD_LIBRARY_PATH into everything

Some Cray modules add paths to CRAY_LD_LIBRARY_PATH
instead of LD_LIBRARY_PATH. This has performance benefits
at load time, but leads to Spack builds not finding their
dependencies from external modules.
Spack now prepends CRAY_LD_LIBRARY_PATH to
LD_LIBRARY_PATH before beginning the build.

* mvapich2: setup cray compilers when on cray

previously, mpich was the only mpi implementation to support
cray systems (because it is the MPI on Cray XC systems). 
Cray cluster systems use mvapich2, which now supports cray
compiler wrappers.

* build_env: clean pkgconf from environment

Cray modules silently add pkgconf to the user environment
This can break builds that do not user pkgconf.
Now we remove it frmo the environment and add it again if it
is in the spec. 

* cray platform: cheat modules for rome/zen2 module on naples/zen node

Cray modules for naples/zen architecture currently specify
rome/zen2. For now, we detect this and return zen for modules
named `craype-x86-rome`.

* compiler: compiler default versions

When detecting compiler default versions for target/compiler
compatibility checks, Spack previously ran the compiler without
setting up its environment. Now we setup a temporary environment
to run the compiler with its modules to detect its version.

* compilers/cce: improve logic to determine C/C++ std flags

* tests: fix existing tests to play nicely with new cray support

* tests: test new functionality

Some new functionality can only be tested on a cray system.
Add tests for what can be tested on a linux system.

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>

2 files changed, 66 insertions, 14 deletions

diff --git a/var/spack/repos/builtin/packages/cray-libsci/package.py b/var/spack/repos/builtin/packages/cray-libsci/package.py
index c0313a1e39..d391f471c3 100755
--- a/var/spack/repos/builtin/packages/cray-libsci/package.py
+++ b/var/spack/repos/builtin/packages/cray-libsci/package.py
@@ -2,10 +2,9 @@
 # Spack Project Developers. See the top-level COPYRIGHT file for details.
 #
 # SPDX-License-Identifier: (Apache-2.0 OR MIT)
-
-from llnl.util.filesystem import LibraryList
-from spack import *
-import os
+from spack.concretize import NoBuildError
+from spack.util.module_cmd import module
+from spack.util.module_cmd import get_path_args_from_module_line
 
 
 class CrayLibsci(Package):
@@ -22,14 +21,53 @@ class CrayLibsci(Package):
     version("16.06.1")
     version("16.03.1")
 
+    variant("shared", default=True, description="enable shared libs")
+    variant("openmp", default=False, description="link with openmp")
+    variant("mpi", default=False, description="link with mpi libs")
+
     provides("blas")
     provides("lapack")
     provides("scalapack")
 
-    # NOTE: Cray compiler wrappers already include linking for the following
+    canonical_names = {
+        'gcc': 'GNU',
+        'cce': 'CRAY',
+        'intel': 'INTEL',
+    }
+
+    @property
+    def modname(self):
+        return "cray-libsci/{0}".format(self.version)
+
+    @property
+    def external_prefix(self):
+        libsci_module = module("show", self.modname).splitlines()
+
+        for line in libsci_module:
+            if "CRAY_LIBSCI_PREFIX_DIR" in line:
+                return get_path_args_from_module_line(line)[0]
+
     @property
     def blas_libs(self):
-        return LibraryList(os.path.join(self.prefix.lib, 'libsci.so'))
+        shared = True if "+shared" in self.spec else False
+        compiler = self.spec.compiler.name
+
+        if "+openmp" in self.spec and "+mpi" in self.spec:
+            lib = "libsci_{0}_mpi_mp"
+        elif "+openmp" in self.spec:
+            lib = "libsci_{0}_mp"
+        elif "+mpi" in self.spec:
+            lib = "libsci_{0}_mpi"
+        else:
+            lib = "libsci_{0}"
+
+        libname = lib.format(self.canonical_names[compiler].lower())
+
+        return find_libraries(
+            libname,
+            root=self.prefix.lib,
+            shared=shared,
+            recursive=False)
 
     @property
     def lapack_libs(self):
diff --git a/var/spack/repos/builtin/packages/mvapich2/package.py b/var/spack/repos/builtin/packages/mvapich2/package.py
index f6301a564a..d8398b336d 100644
--- a/var/spack/repos/builtin/packages/mvapich2/package.py
+++ b/var/spack/repos/builtin/packages/mvapich2/package.py
@@ -208,10 +208,17 @@ class Mvapich2(AutotoolsPackage):
             env.set('SLURM_MPI_TYPE', 'pmi2')
 
     def setup_dependent_build_environment(self, env, dependent_spec):
-        env.set('MPICC', os.path.join(self.prefix.bin, 'mpicc'))
-        env.set('MPICXX', os.path.join(self.prefix.bin, 'mpicxx'))
-        env.set('MPIF77', os.path.join(self.prefix.bin, 'mpif77'))
-        env.set('MPIF90', os.path.join(self.prefix.bin, 'mpif90'))
+        # On Cray, the regular compiler wrappers *are* the MPI wrappers.
+        if 'platform=cray' in self.spec:
+            env.set('MPICC',  spack_cc)
+            env.set('MPICXX', spack_cxx)
+            env.set('MPIF77', spack_fc)
+            env.set('MPIF90', spack_fc)
+        else:
+            env.set('MPICC',  join_path(self.prefix.bin, 'mpicc'))
+            env.set('MPICXX', join_path(self.prefix.bin, 'mpicxx'))
+            env.set('MPIF77', join_path(self.prefix.bin, 'mpif77'))
+            env.set('MPIF90', join_path(self.prefix.bin, 'mpif90'))
 
         env.set('MPICH_CC', spack_cc)
         env.set('MPICH_CXX', spack_cxx)
@@ -220,10 +227,17 @@ class Mvapich2(AutotoolsPackage):
         env.set('MPICH_FC', spack_fc)
 
     def setup_dependent_package(self, module, dependent_spec):
-        self.spec.mpicc  = os.path.join(self.prefix.bin, 'mpicc')
-        self.spec.mpicxx = os.path.join(self.prefix.bin, 'mpicxx')
-        self.spec.mpifc  = os.path.join(self.prefix.bin, 'mpif90')
-        self.spec.mpif77 = os.path.join(self.prefix.bin, 'mpif77')
+        if 'platform=cray' in self.spec:
+            self.spec.mpicc = spack_cc
+            self.spec.mpicxx = spack_cxx
+            self.spec.mpifc = spack_fc
+            self.spec.mpif77 = spack_f77
+        else:
+            self.spec.mpicc  = join_path(self.prefix.bin, 'mpicc')
+            self.spec.mpicxx = join_path(self.prefix.bin, 'mpicxx')
+            self.spec.mpifc  = join_path(self.prefix.bin, 'mpif90')
+            self.spec.mpif77 = join_path(self.prefix.bin, 'mpif77')
+
         self.spec.mpicxx_shared_libs = [
             os.path.join(self.prefix.lib, 'libmpicxx.{0}'.format(dso_suffix)),
             os.path.join(self.prefix.lib, 'libmpi.{0}'.format(dso_suffix))