Add new RubyPackage build system base class (#18199)

* Add new RubyPackage build system base class

* Ruby: add spack external find support

* Add build tests for RubyPackage

5 files changed, 275 insertions, 8 deletions

diff --git a/lib/spack/docs/build_systems/rubypackage.rst b/lib/spack/docs/build_systems/rubypackage.rst
index 6310507809..d1deaf6fa7 100644
--- a/lib/spack/docs/build_systems/rubypackage.rst
+++ b/lib/spack/docs/build_systems/rubypackage.rst
@@ -12,5 +12,173 @@ RubyPackage
 Like Perl, Python, and R, Ruby has its own build system for
 installing Ruby gems.
 
-This build system is a work-in-progress. See
-https://github.com/spack/spack/pull/3127 for more information.
+^^^^^^
+Phases
+^^^^^^
+
+The ``RubyPackage`` base class provides the following phases that
+can be overridden:
+
+#. ``build`` - build everything needed to install
+#. ``install`` - install everything from build directory
+
+For packages that come with a ``*.gemspec`` file, these phases run:
+
+.. code-block:: console
+
+   $ gem build *.gemspec
+   $ gem install *.gem
+
+
+For packages that come with a ``Rakefile`` file, these phases run:
+
+.. code-block:: console
+
+   $ rake package
+   $ gem install *.gem
+
+
+For packages that come pre-packaged as a ``*.gem`` file, the build
+phase is skipped and the install phase runs:
+
+.. code-block:: console
+
+   $ gem install *.gem
+
+
+These are all standard ``gem`` commands and can be found by running:
+
+.. code-block:: console
+
+   $ gem help commands
+
+
+For packages that only distribute ``*.gem`` files, these files can be
+downloaded with the ``expand=False`` option in the ``version`` directive.
+The build phase will be automatically skipped.
+
+^^^^^^^^^^^^^^^
+Important files
+^^^^^^^^^^^^^^^
+
+When building from source, Ruby packages can be identified by the
+presence of any of the following files:
+
+* ``*.gemspec``
+* ``Rakefile``
+* ``setup.rb`` (not yet supported)
+
+However, not all Ruby packages are released as source code. Some are only
+released as ``*.gem`` files. These files can be extracted using:
+
+.. code-block:: console
+
+   $ gem unpack *.gem
+
+
+^^^^^^^^^^^
+Description
+^^^^^^^^^^^
+
+The ``*.gemspec`` file may contain something like:
+
+.. code-block:: ruby
+
+   summary = 'An implementation of the AsciiDoc text processor and publishing toolchain'
+   description = 'A fast, open source text processor and publishing toolchain for converting AsciiDoc content to HTML 5, DocBook 5, and other formats.'
+
+
+Either of these can be used for the description of the Spack package.
+
+^^^^^^^^
+Homepage
+^^^^^^^^
+
+The ``*.gemspec`` file may contain something like:
+
+.. code-block:: ruby
+
+   homepage = 'https://asciidoctor.org'
+
+
+This should be used as the official homepage of the Spack package.
+
+^^^^^^^^^^^^^^^^^^^^^^^^^
+Build system dependencies
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+All Ruby packages require Ruby at build and run-time. For this reason,
+the base class contains:
+
+.. code-block:: python
+
+   extends('ruby')
+   depends_on('ruby', type=('build', 'run'))
+
+
+The ``*.gemspec`` file may contain something like:
+
+.. code-block:: ruby
+
+   required_ruby_version = '>= 2.3.0'
+
+
+This can be added to the Spack package using:
+
+.. code-block:: python
+
+   depends_on('ruby@2.3.0:', type=('build', 'run'))
+
+
+^^^^^^^^^^^^^^^^^
+Ruby dependencies
+^^^^^^^^^^^^^^^^^
+
+When you install a package with ``gem``, it reads the ``*.gemspec``
+file in order to determine the dependencies of the package.
+If the dependencies are not yet installed, ``gem`` downloads them
+and installs them for you. This may sound convenient, but Spack
+cannot rely on this behavior for two reasons:
+
+#. Spack needs to be able to install packages on air-gapped networks.
+
+   If there is no internet connection, ``gem`` can't download the
+   package dependencies. By explicitly listing every dependency in
+   the ``package.py``, Spack knows what to download ahead of time.
+
+#. Duplicate installations of the same dependency may occur.
+
+   Spack supports *activation* of Ruby extensions, which involves
+   symlinking the package installation prefix to the Ruby installation
+   prefix. If your package is missing a dependency, that dependency
+   will be installed to the installation directory of the same package.
+   If you try to activate the package + dependency, it may cause a
+   problem if that package has already been activated.
+
+For these reasons, you must always explicitly list all dependencies.
+Although the documentation may list the package's dependencies,
+often the developers assume people will use ``gem`` and won't have to
+worry about it. Always check the ``*.gemspec`` file to find the true
+dependencies.
+
+Check for the following clues in the ``*.gemspec`` file:
+
+* ``add_runtime_dependency``
+
+  These packages are required for installation.
+
+* ``add_dependency``
+
+  This is an alias for ``add_runtime_dependency``
+
+* ``add_development_dependency``
+
+  These packages are optional dependencies used for development.
+  They should not be added as dependencies of the package.
+
+^^^^^^^^^^^^^^^^^^^^^^
+External documentation
+^^^^^^^^^^^^^^^^^^^^^^
+
+For more information on Ruby packaging, see:
+https://guides.rubygems.org/
diff --git a/lib/spack/spack/build_systems/ruby.py b/lib/spack/spack/build_systems/ruby.py
new file mode 100644
index 0000000000..32d6633164
--- /dev/null
+++ b/lib/spack/spack/build_systems/ruby.py
@@ -0,0 +1,59 @@
+# Copyright 2013-2020 Lawrence Livermore National Security, LLC and other
+# Spack Project Developers. See the top-level COPYRIGHT file for details.
+#
+# SPDX-License-Identifier: (Apache-2.0 OR MIT)
+
+import glob
+import inspect
+
+from spack.directives import depends_on, extends
+from spack.package import PackageBase, run_after
+
+
+class RubyPackage(PackageBase):
+    """Specialized class for building Ruby gems.
+
+    This class provides two phases that can be overridden if required:
+
+    #. :py:meth:`~.RubyPackage.build`
+    #. :py:meth:`~.RubyPackage.install`
+    """
+    #: Phases of a Ruby package
+    phases = ['build', 'install']
+
+    #: This attribute is used in UI queries that need to know the build
+    #: system base class
+    build_system_class = 'RubyPackage'
+
+    extends('ruby')
+
+    depends_on('ruby', type=('build', 'run'))
+
+    def build(self, spec, prefix):
+        """Build a Ruby gem."""
+
+        # ruby-rake provides both rake.gemspec and Rakefile, but only
+        # rake.gemspec can be built without an existing rake installation
+        gemspecs = glob.glob('*.gemspec')
+        rakefiles = glob.glob('Rakefile')
+        if gemspecs:
+            inspect.getmodule(self).gem('build', '--norc', gemspecs[0])
+        elif rakefiles:
+            jobs = inspect.getmodule(self).make_jobs
+            inspect.getmodule(self).rake('package', '-j{0}'.format(jobs))
+        else:
+            # Some Ruby packages only ship `*.gem` files, so nothing to build
+            pass
+
+    def install(self, spec, prefix):
+        """Install a Ruby gem.
+
+        The ruby package sets ``GEM_HOME`` to tell gem where to install to."""
+
+        gems = glob.glob('*.gem')
+        if gems:
+            inspect.getmodule(self).gem(
+                'install', '--norc', '--ignore-dependencies', gems[0])
+
+    # Check that self.prefix is there after installation
+    run_after('install')(PackageBase.sanity_check_prefix)
diff --git a/lib/spack/spack/cmd/create.py b/lib/spack/spack/cmd/create.py
index 48326868ae..57d4bd7ded 100644
--- a/lib/spack/spack/cmd/create.py
+++ b/lib/spack/spack/cmd/create.py
@@ -352,6 +352,34 @@ class OctavePackageTemplate(PackageTemplate):
         super(OctavePackageTemplate, self).__init__(name, *args, **kwargs)
 
 
+class RubyPackageTemplate(PackageTemplate):
+    """Provides appropriate overrides for Ruby packages"""
+
+    base_class_name = 'RubyPackage'
+
+    dependencies = """\
+    # FIXME: Add dependencies if required. Only add the ruby dependency
+    # if you need specific versions. A generic ruby dependency is
+    # added implicity by the RubyPackage class.
+    # depends_on('ruby@X.Y.Z:', type=('build', 'run'))
+    # depends_on('ruby-foo', type=('build', 'run'))"""
+
+    body_def = """\
+    def build(self, spec, prefix):
+        # FIXME: If not needed delete this function
+        pass"""
+
+    def __init__(self, name, *args, **kwargs):
+        # If the user provided `--name ruby-numpy`, don't rename it
+        # ruby-ruby-numpy
+        if not name.startswith('ruby-'):
+            # Make it more obvious that we are renaming the package
+            tty.msg("Changing package name from {0} to ruby-{0}".format(name))
+            name = 'ruby-{0}'.format(name)
+
+        super(RubyPackageTemplate, self).__init__(name, *args, **kwargs)
+
+
 class MakefilePackageTemplate(PackageTemplate):
     """Provides appropriate overrides for Makefile packages"""
 
@@ -410,6 +438,7 @@ templates = {
     'perlmake':   PerlmakePackageTemplate,
     'perlbuild':  PerlbuildPackageTemplate,
     'octave':     OctavePackageTemplate,
+    'ruby':       RubyPackageTemplate,
     'makefile':   MakefilePackageTemplate,
     'intel':      IntelPackageTemplate,
     'meson':      MesonPackageTemplate,
@@ -464,12 +493,16 @@ class BuildSystemGuesser:
         """Try to guess the type of build system used by a project based on
         the contents of its archive or the URL it was downloaded from."""
 
-        # Most octave extensions are hosted on Octave-Forge:
-        #     https://octave.sourceforge.net/index.html
-        # They all have the same base URL.
-        if url is not None and 'downloads.sourceforge.net/octave/' in url:
-            self.build_system = 'octave'
-            return
+        if url is not None:
+            # Most octave extensions are hosted on Octave-Forge:
+            #     https://octave.sourceforge.net/index.html
+            # They all have the same base URL.
+            if 'downloads.sourceforge.net/octave/' in url:
+                self.build_system = 'octave'
+                return
+            if url.endswith('.gem'):
+                self.build_system = 'ruby'
+                return
 
         # A list of clues that give us an idea of the build system a package
         # uses. If the regular expression matches a file contained in the
@@ -488,6 +521,9 @@ class BuildSystemGuesser:
             (r'/WORKSPACE$',          'bazel'),
             (r'/Build\.PL$',          'perlbuild'),
             (r'/Makefile\.PL$',       'perlmake'),
+            (r'/.*\.gemspec$',        'ruby'),
+            (r'/Rakefile$',           'ruby'),
+            (r'/setup\.rb$',          'ruby'),
             (r'/.*\.pro$',            'qmake'),
             (r'/(GNU)?[Mm]akefile$',  'makefile'),
             (r'/DESCRIPTION$',        'octave'),
diff --git a/lib/spack/spack/pkgkit.py b/lib/spack/spack/pkgkit.py
index e2a29894f7..5ffec6caef 100644
--- a/lib/spack/spack/pkgkit.py
+++ b/lib/spack/spack/pkgkit.py
@@ -27,6 +27,7 @@ from spack.build_systems.octave import OctavePackage
 from spack.build_systems.python import PythonPackage
 from spack.build_systems.r import RPackage
 from spack.build_systems.perl import PerlPackage
+from spack.build_systems.ruby import RubyPackage
 from spack.build_systems.intel import IntelPackage
 from spack.build_systems.meson import MesonPackage
 from spack.build_systems.sip import SIPPackage
diff --git a/lib/spack/spack/test/build_system_guess.py b/lib/spack/spack/test/build_system_guess.py
index 76197e9e15..64b37e546e 100644
--- a/lib/spack/spack/test/build_system_guess.py
+++ b/lib/spack/spack/test/build_system_guess.py
@@ -23,6 +23,9 @@ import spack.stage
         ('WORKSPACE',      'bazel'),
         ('Makefile.PL',    'perlmake'),
         ('Build.PL',       'perlbuild'),
+        ('foo.gemspec',    'ruby'),
+        ('Rakefile',       'ruby'),
+        ('setup.rb',       'ruby'),
         ('GNUmakefile',    'makefile'),
         ('makefile',       'makefile'),
         ('Makefile',       'makefile'),