diff options
Diffstat (limited to 'lib')
80 files changed, 6226 insertions, 2323 deletions
diff --git a/lib/spack/docs/build_systems.rst b/lib/spack/docs/build_systems.rst index 12191d29f0..1ce8d6746e 100644 --- a/lib/spack/docs/build_systems.rst +++ b/lib/spack/docs/build_systems.rst @@ -65,7 +65,6 @@ on these ideas for each distinct build system that Spack supports: build_systems/custompackage build_systems/inteloneapipackage build_systems/intelpackage - build_systems/multiplepackage build_systems/rocmpackage build_systems/sourceforgepackage diff --git a/lib/spack/docs/build_systems/autotoolspackage.rst b/lib/spack/docs/build_systems/autotoolspackage.rst index d341d28d08..88fabc0c5d 100644 --- a/lib/spack/docs/build_systems/autotoolspackage.rst +++ b/lib/spack/docs/build_systems/autotoolspackage.rst @@ -5,9 +5,9 @@ .. _autotoolspackage: ----------------- -AutotoolsPackage ----------------- +--------- +Autotools +--------- Autotools is a GNU build system that provides a build-script generator. By running the platform-independent ``./configure`` script that comes @@ -17,7 +17,7 @@ with the package, you can generate a platform-dependent Makefile. Phases ^^^^^^ -The ``AutotoolsPackage`` base class comes with the following phases: +The ``AutotoolsBuilder`` and ``AutotoolsPackage`` base classes come with the following phases: #. ``autoreconf`` - generate the configure script #. ``configure`` - generate the Makefiles diff --git a/lib/spack/docs/build_systems/bundlepackage.rst b/lib/spack/docs/build_systems/bundlepackage.rst index 8787dce546..7a826f5e17 100644 --- a/lib/spack/docs/build_systems/bundlepackage.rst +++ b/lib/spack/docs/build_systems/bundlepackage.rst @@ -5,9 +5,9 @@ .. _bundlepackage: -------------- -BundlePackage -------------- +------ +Bundle +------ ``BundlePackage`` represents a set of packages that are expected to work well together, such as a collection of commonly used software libraries. The diff --git a/lib/spack/docs/build_systems/cmakepackage.rst b/lib/spack/docs/build_systems/cmakepackage.rst index 9544a7df73..7a1db842de 100644 --- a/lib/spack/docs/build_systems/cmakepackage.rst +++ b/lib/spack/docs/build_systems/cmakepackage.rst @@ -5,9 +5,9 @@ .. _cmakepackage: ------------- -CMakePackage ------------- +----- +CMake +----- Like Autotools, CMake is a widely-used build-script generator. Designed by Kitware, CMake is the most popular build system for new C, C++, and @@ -21,7 +21,7 @@ whereas Autotools is Unix-only. Phases ^^^^^^ -The ``CMakePackage`` base class comes with the following phases: +The ``CMakeBuilder`` and ``CMakePackage`` base classes come with the following phases: #. ``cmake`` - generate the Makefile #. ``build`` - build the package @@ -130,8 +130,8 @@ Adding flags to cmake To add additional flags to the ``cmake`` call, simply override the ``cmake_args`` function. The following example defines values for the flags ``WHATEVER``, ``ENABLE_BROKEN_FEATURE``, ``DETECT_HDF5``, and ``THREADS`` with -and without the :meth:`~spack.build_systems.cmake.CMakePackage.define` and -:meth:`~spack.build_systems.cmake.CMakePackage.define_from_variant` helper functions: +and without the :meth:`~spack.build_systems.cmake.CMakeBuilder.define` and +:meth:`~spack.build_systems.cmake.CMakeBuilder.define_from_variant` helper functions: .. code-block:: python diff --git a/lib/spack/docs/build_systems/luapackage.rst b/lib/spack/docs/build_systems/luapackage.rst index 6332edfc20..fd70f90c49 100644 --- a/lib/spack/docs/build_systems/luapackage.rst +++ b/lib/spack/docs/build_systems/luapackage.rst @@ -5,11 +5,11 @@ .. _luapackage: ------------- -LuaPackage ------------- +--- +Lua +--- -LuaPackage is a helper for the common case of Lua packages that provide +The ``Lua`` build-system is a helper for the common case of Lua packages that provide a rockspec file. This is not meant to take a rock archive, but to build a source archive or repository that provides a rockspec, which should cover most lua packages. In the case a Lua package builds by Make rather than @@ -19,7 +19,7 @@ luarocks, prefer MakefilePackage. Phases ^^^^^^ -The ``LuaPackage`` base class comes with the following phases: +The ``LuaBuilder`` and `LuaPackage`` base classes come with the following phases: #. ``unpack`` - if using a rock, unpacks the rock and moves into the source directory #. ``preprocess`` - adjust sources or rockspec to fix build diff --git a/lib/spack/docs/build_systems/makefilepackage.rst b/lib/spack/docs/build_systems/makefilepackage.rst index c092432037..5a83d612fa 100644 --- a/lib/spack/docs/build_systems/makefilepackage.rst +++ b/lib/spack/docs/build_systems/makefilepackage.rst @@ -5,9 +5,9 @@ .. _makefilepackage: ---------------- -MakefilePackage ---------------- +-------- +Makefile +-------- The most primitive build system a package can use is a plain Makefile. Makefiles are simple to write for small projects, but they usually @@ -18,7 +18,7 @@ variables. Phases ^^^^^^ -The ``MakefilePackage`` base class comes with 3 phases: +The ``MakefileBuilder`` and ``MakefilePackage`` base classes come with 3 phases: #. ``edit`` - edit the Makefile #. ``build`` - build the project diff --git a/lib/spack/docs/build_systems/mavenpackage.rst b/lib/spack/docs/build_systems/mavenpackage.rst index 94ce128d3a..d1237ce34c 100644 --- a/lib/spack/docs/build_systems/mavenpackage.rst +++ b/lib/spack/docs/build_systems/mavenpackage.rst @@ -5,9 +5,9 @@ .. _mavenpackage: ------------- -MavenPackage ------------- +----- +Maven +----- Apache Maven is a general-purpose build system that does not rely on Makefiles to build software. It is designed for building and @@ -17,7 +17,7 @@ managing and Java-based project. Phases ^^^^^^ -The ``MavenPackage`` base class comes with the following phases: +The ``MavenBuilder`` and ``MavenPackage`` base classes come with the following phases: #. ``build`` - compile code and package into a JAR file #. ``install`` - copy to installation prefix diff --git a/lib/spack/docs/build_systems/mesonpackage.rst b/lib/spack/docs/build_systems/mesonpackage.rst index 5ca444dcb1..c32b2241bc 100644 --- a/lib/spack/docs/build_systems/mesonpackage.rst +++ b/lib/spack/docs/build_systems/mesonpackage.rst @@ -5,9 +5,9 @@ .. _mesonpackage: ------------- -MesonPackage ------------- +----- +Meson +----- Much like Autotools and CMake, Meson is a build system. But it is meant to be both fast and as user friendly as possible. GNOME's goal @@ -17,7 +17,7 @@ is to port modules to use the Meson build system. Phases ^^^^^^ -The ``MesonPackage`` base class comes with the following phases: +The ``MesonBuilder`` and ``MesonPackage`` base classes come with the following phases: #. ``meson`` - generate ninja files #. ``build`` - build the project diff --git a/lib/spack/docs/build_systems/multiplepackage.rst b/lib/spack/docs/build_systems/multiplepackage.rst deleted file mode 100644 index 71751f0dbf..0000000000 --- a/lib/spack/docs/build_systems/multiplepackage.rst +++ /dev/null @@ -1,350 +0,0 @@ -.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other - Spack Project Developers. See the top-level COPYRIGHT file for details. - - SPDX-License-Identifier: (Apache-2.0 OR MIT) - -.. _multiplepackage: - ----------------------- -Multiple Build Systems ----------------------- - -Quite frequently, a package will change build systems from one version to the -next. For example, a small project that once used a single Makefile to build -may now require Autotools to handle the increased number of files that need to -be compiled. Or, a package that once used Autotools may switch to CMake for -Windows support. In this case, it becomes a bit more challenging to write a -single build recipe for this package in Spack. - -There are several ways that this can be handled in Spack: - -#. Subclass the new build system, and override phases as needed (preferred) -#. Subclass ``Package`` and implement ``install`` as needed -#. Create separate ``*-cmake``, ``*-autotools``, etc. packages for each build system -#. Rename the old package to ``*-legacy`` and create a new package -#. Move the old package to a ``legacy`` repository and create a new package -#. Drop older versions that only support the older build system - -Of these options, 1 is preferred, and will be demonstrated in this -documentation. Options 3-5 have issues with concretization, so shouldn't be -used. Options 4-5 also don't support more than two build systems. Option 6 only -works if the old versions are no longer needed. Option 1 is preferred over 2 -because it makes it easier to drop the old build system entirely. - -The exact syntax of the package depends on which build systems you need to -support. Below are a couple of common examples. - -^^^^^^^^^^^^^^^^^^^^^ -Makefile -> Autotools -^^^^^^^^^^^^^^^^^^^^^ - -Let's say we have the following package: - -.. code-block:: python - - class Foo(MakefilePackage): - version("1.2.0", sha256="...") - - def edit(self, spec, prefix): - filter_file("CC=", "CC=" + spack_cc, "Makefile") - - def install(self, spec, prefix): - install_tree(".", prefix) - - -The package subclasses from :ref:`makefilepackage`, which has three phases: - -#. ``edit`` (does nothing by default) -#. ``build`` (runs ``make`` by default) -#. ``install`` (runs ``make install`` by default) - -In this case, the ``install`` phase needed to be overridden because the -Makefile did not have an install target. We also modify the Makefile to use -Spack's compiler wrappers. The default ``build`` phase is not changed. - -Starting with version 1.3.0, we want to use Autotools to build instead. -:ref:`autotoolspackage` has four phases: - -#. ``autoreconf`` (does not if a configure script already exists) -#. ``configure`` (runs ``./configure --prefix=...`` by default) -#. ``build`` (runs ``make`` by default) -#. ``install`` (runs ``make install`` by default) - -If the only version we need to support is 1.3.0, the package would look as -simple as: - -.. code-block:: python - - class Foo(AutotoolsPackage): - version("1.3.0", sha256="...") - - def configure_args(self): - return ["--enable-shared"] - - -In this case, we use the default methods for each phase and only override -``configure_args`` to specify additional flags to pass to ``./configure``. - -If we wanted to write a single package that supports both versions 1.2.0 and -1.3.0, it would look something like: - -.. code-block:: python - - class Foo(AutotoolsPackage): - version("1.3.0", sha256="...") - version("1.2.0", sha256="...", deprecated=True) - - def configure_args(self): - return ["--enable-shared"] - - # Remove the following once version 1.2.0 is dropped - @when("@:1.2") - def patch(self): - filter_file("CC=", "CC=" + spack_cc, "Makefile") - - @when("@:1.2") - def autoreconf(self, spec, prefix): - pass - - @when("@:1.2") - def configure(self, spec, prefix): - pass - - @when("@:1.2") - def install(self, spec, prefix): - install_tree(".", prefix) - - -There are a few interesting things to note here: - -* We added ``deprecated=True`` to version 1.2.0. This signifies that version - 1.2.0 is deprecated and shouldn't be used. However, if a user still relies - on version 1.2.0, it's still there and builds just fine. -* We moved the contents of the ``edit`` phase to the ``patch`` function. Since - ``AutotoolsPackage`` doesn't have an ``edit`` phase, the only way for this - step to be executed is to move it to the ``patch`` function, which always - gets run. -* The ``autoreconf`` and ``configure`` phases become no-ops. Since the old - Makefile-based build system doesn't use these, we ignore these phases when - building ``foo@1.2.0``. -* The ``@when`` decorator is used to override these phases only for older - versions. The default methods are used for ``foo@1.3:``. - -Once a new Spack release comes out, version 1.2.0 and everything below the -comment can be safely deleted. The result is the same as if we had written a -package for version 1.3.0 from scratch. - -^^^^^^^^^^^^^^^^^^ -Autotools -> CMake -^^^^^^^^^^^^^^^^^^ - -Let's say we have the following package: - -.. code-block:: python - - class Bar(AutotoolsPackage): - version("1.2.0", sha256="...") - - def configure_args(self): - return ["--enable-shared"] - - -The package subclasses from :ref:`autotoolspackage`, which has four phases: - -#. ``autoreconf`` (does not if a configure script already exists) -#. ``configure`` (runs ``./configure --prefix=...`` by default) -#. ``build`` (runs ``make`` by default) -#. ``install`` (runs ``make install`` by default) - -In this case, we use the default methods for each phase and only override -``configure_args`` to specify additional flags to pass to ``./configure``. - -Starting with version 1.3.0, we want to use CMake to build instead. -:ref:`cmakepackage` has three phases: - -#. ``cmake`` (runs ``cmake ...`` by default) -#. ``build`` (runs ``make`` by default) -#. ``install`` (runs ``make install`` by default) - -If the only version we need to support is 1.3.0, the package would look as -simple as: - -.. code-block:: python - - class Bar(CMakePackage): - version("1.3.0", sha256="...") - - def cmake_args(self): - return [self.define("BUILD_SHARED_LIBS", True)] - - -In this case, we use the default methods for each phase and only override -``cmake_args`` to specify additional flags to pass to ``cmake``. - -If we wanted to write a single package that supports both versions 1.2.0 and -1.3.0, it would look something like: - -.. code-block:: python - - class Bar(CMakePackage): - version("1.3.0", sha256="...") - version("1.2.0", sha256="...", deprecated=True) - - def cmake_args(self): - return [self.define("BUILD_SHARED_LIBS", True)] - - # Remove the following once version 1.2.0 is dropped - def configure_args(self): - return ["--enable-shared"] - - @when("@:1.2") - def cmake(self, spec, prefix): - configure("--prefix=" + prefix, *self.configure_args()) - - -There are a few interesting things to note here: - -* We added ``deprecated=True`` to version 1.2.0. This signifies that version - 1.2.0 is deprecated and shouldn't be used. However, if a user still relies - on version 1.2.0, it's still there and builds just fine. -* Since CMake and Autotools are so similar, we only need to override the - ``cmake`` phase, we can use the default ``build`` and ``install`` phases. -* We override ``cmake`` to run ``./configure`` for older versions. - ``configure_args`` remains the same. -* The ``@when`` decorator is used to override these phases only for older - versions. The default methods are used for ``bar@1.3:``. - -Once a new Spack release comes out, version 1.2.0 and everything below the -comment can be safely deleted. The result is the same as if we had written a -package for version 1.3.0 from scratch. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Multiple build systems for the same version -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -During the transition from one build system to another, developers often -support multiple build systems at the same time. Spack can only use a single -build system for a single version. To decide which build system to use for a -particular version, take the following things into account: - -1. If the developers explicitly state that one build system is preferred over - another, use that one. -2. If one build system is considered "experimental" while another is considered - "stable", use the stable build system. -3. Otherwise, use the newer build system. - -The developer preference for which build system to use can change over time as -a newer build system becomes stable/recommended. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Dropping support for old build systems -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -When older versions of a package don't support a newer build system, it can be -tempting to simply delete them from a package. This significantly reduces -package complexity and makes the build recipe much easier to maintain. However, -other packages or Spack users may rely on these older versions. The recommended -approach is to first support both build systems (as demonstrated above), -:ref:`deprecate <deprecate>` versions that rely on the old build system, and -remove those versions and any phases that needed to be overridden in the next -Spack release. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Three or more build systems -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In rare cases, a package may change build systems multiple times. For example, -a package may start with Makefiles, then switch to Autotools, then switch to -CMake. The same logic used above can be extended to any number of build systems. -For example: - -.. code-block:: python - - class Baz(CMakePackage): - version("1.4.0", sha256="...") # CMake - version("1.3.0", sha256="...") # Autotools - version("1.2.0", sha256="...") # Makefile - - def cmake_args(self): - return [self.define("BUILD_SHARED_LIBS", True)] - - # Remove the following once version 1.3.0 is dropped - def configure_args(self): - return ["--enable-shared"] - - @when("@1.3") - def cmake(self, spec, prefix): - configure("--prefix=" + prefix, *self.configure_args()) - - # Remove the following once version 1.2.0 is dropped - @when("@:1.2") - def patch(self): - filter_file("CC=", "CC=" + spack_cc, "Makefile") - - @when("@:1.2") - def cmake(self, spec, prefix): - pass - - @when("@:1.2") - def install(self, spec, prefix): - install_tree(".", prefix) - - -^^^^^^^^^^^^^^^^^^^ -Additional examples -^^^^^^^^^^^^^^^^^^^ - -When writing new packages, it often helps to see examples of existing packages. -Here is an incomplete list of existing Spack packages that have changed build -systems before: - -================ ===================== ================ -Package Previous Build System New Build System -================ ===================== ================ -amber custom CMake -arpack-ng Autotools CMake -atk Autotools Meson -blast None Autotools -dyninst Autotools CMake -evtgen Autotools CMake -fish Autotools CMake -gdk-pixbuf Autotools Meson -glib Autotools Meson -glog Autotools CMake -gmt Autotools CMake -gtkplus Autotools Meson -hpl Makefile Autotools -interproscan Perl Maven -jasper Autotools CMake -kahip SCons CMake -kokkos Makefile CMake -kokkos-kernels Makefile CMake -leveldb Makefile CMake -libdrm Autotools Meson -libjpeg-turbo Autotools CMake -mesa Autotools Meson -metis None CMake -mpifileutils Autotools CMake -muparser Autotools CMake -mxnet Makefile CMake -nest Autotools CMake -neuron Autotools CMake -nsimd CMake nsconfig -opennurbs Makefile CMake -optional-lite None CMake -plasma Makefile CMake -preseq Makefile Autotools -protobuf Autotools CMake -py-pygobject Autotools Python -singularity Autotools Makefile -span-lite None CMake -ssht Makefile CMake -string-view-lite None CMake -superlu Makefile CMake -superlu-dist Makefile CMake -uncrustify Autotools CMake -================ ===================== ================ - -Packages that support multiple build systems can be a bit confusing to write. -Don't hesitate to open an issue or draft pull request and ask for advice from -other Spack developers! diff --git a/lib/spack/docs/build_systems/octavepackage.rst b/lib/spack/docs/build_systems/octavepackage.rst index 9a81671db6..32e8cb61b3 100644 --- a/lib/spack/docs/build_systems/octavepackage.rst +++ b/lib/spack/docs/build_systems/octavepackage.rst @@ -5,9 +5,9 @@ .. _octavepackage: -------------- -OctavePackage -------------- +------ +Octave +------ Octave has its own build system for installing packages. @@ -15,7 +15,7 @@ Octave has its own build system for installing packages. Phases ^^^^^^ -The ``OctavePackage`` base class has a single phase: +The ``OctaveBuilder`` and ``OctavePackage`` base classes have a single phase: #. ``install`` - install the package diff --git a/lib/spack/docs/build_systems/perlpackage.rst b/lib/spack/docs/build_systems/perlpackage.rst index be81ca6ce9..4e1f613c3b 100644 --- a/lib/spack/docs/build_systems/perlpackage.rst +++ b/lib/spack/docs/build_systems/perlpackage.rst @@ -5,9 +5,9 @@ .. _perlpackage: ------------ -PerlPackage ------------ +---- +Perl +---- Much like Octave, Perl has its own language-specific build system. @@ -16,7 +16,7 @@ build system. Phases ^^^^^^ -The ``PerlPackage`` base class comes with 3 phases that can be overridden: +The ``PerlBuilder`` and ``PerlPackage`` base classes come with 3 phases that can be overridden: #. ``configure`` - configure the package #. ``build`` - build the package diff --git a/lib/spack/docs/build_systems/qmakepackage.rst b/lib/spack/docs/build_systems/qmakepackage.rst index 6e8bcef7cc..215d59536e 100644 --- a/lib/spack/docs/build_systems/qmakepackage.rst +++ b/lib/spack/docs/build_systems/qmakepackage.rst @@ -5,9 +5,9 @@ .. _qmakepackage: ------------- -QMakePackage ------------- +----- +QMake +----- Much like Autotools and CMake, QMake is a build-script generator designed by the developers of Qt. In its simplest form, Spack's @@ -29,7 +29,7 @@ variables or edit ``*.pro`` files to get things working properly. Phases ^^^^^^ -The ``QMakePackage`` base class comes with the following phases: +The ``QMakeBuilder`` and ``QMakePackage`` base classes come with the following phases: #. ``qmake`` - generate Makefiles #. ``build`` - build the project diff --git a/lib/spack/docs/build_systems/racketpackage.rst b/lib/spack/docs/build_systems/racketpackage.rst index 8ba37ceeba..5e09ffca4a 100644 --- a/lib/spack/docs/build_systems/racketpackage.rst +++ b/lib/spack/docs/build_systems/racketpackage.rst @@ -5,9 +5,9 @@ .. _racketpackage: -------------- -RacketPackage -------------- +------ +Racket +------ Much like Python, Racket packages and modules have their own special build system. To learn more about the specifics of Racket package system, please refer to the @@ -17,7 +17,7 @@ To learn more about the specifics of Racket package system, please refer to the Phases ^^^^^^ -The ``RacketPackage`` base class provides an ``install`` phase that +The ``RacketBuilder`` and ``RacketPackage`` base classes provides an ``install`` phase that can be overridden, corresponding to the use of: .. code-block:: console diff --git a/lib/spack/docs/build_systems/rpackage.rst b/lib/spack/docs/build_systems/rpackage.rst index 671af779b1..ebf2270e8e 100644 --- a/lib/spack/docs/build_systems/rpackage.rst +++ b/lib/spack/docs/build_systems/rpackage.rst @@ -19,7 +19,7 @@ new Spack packages for. Phases ^^^^^^ -The ``RPackage`` base class has a single phase: +The ``RBuilder`` and ``RPackage`` base classes have a single phase: #. ``install`` - install the package diff --git a/lib/spack/docs/build_systems/rubypackage.rst b/lib/spack/docs/build_systems/rubypackage.rst index b64ac60b2f..5b6ec462a6 100644 --- a/lib/spack/docs/build_systems/rubypackage.rst +++ b/lib/spack/docs/build_systems/rubypackage.rst @@ -5,9 +5,9 @@ .. _rubypackage: ------------ -RubyPackage ------------ +---- +Ruby +---- Like Perl, Python, and R, Ruby has its own build system for installing Ruby gems. @@ -16,7 +16,7 @@ installing Ruby gems. Phases ^^^^^^ -The ``RubyPackage`` base class provides the following phases that +The ``RubyBuilder`` and ``RubyPackage`` base classes provide the following phases that can be overridden: #. ``build`` - build everything needed to install diff --git a/lib/spack/docs/build_systems/sconspackage.rst b/lib/spack/docs/build_systems/sconspackage.rst index cea0408651..aea5dacfa7 100644 --- a/lib/spack/docs/build_systems/sconspackage.rst +++ b/lib/spack/docs/build_systems/sconspackage.rst @@ -5,9 +5,9 @@ .. _sconspackage: ------------- -SConsPackage ------------- +----- +SCons +----- SCons is a general-purpose build system that does not rely on Makefiles to build software. SCons is written in Python, and handles @@ -42,7 +42,7 @@ As previously mentioned, SCons allows developers to add subcommands like $ scons install -To facilitate this, the ``SConsPackage`` base class provides the +To facilitate this, the ``SConsBuilder`` and ``SconsPackage`` base classes provide the following phases: #. ``build`` - build the package diff --git a/lib/spack/docs/build_systems/sippackage.rst b/lib/spack/docs/build_systems/sippackage.rst index 5235015a92..3e77968e80 100644 --- a/lib/spack/docs/build_systems/sippackage.rst +++ b/lib/spack/docs/build_systems/sippackage.rst @@ -5,9 +5,9 @@ .. _sippackage: ----------- -SIPPackage ----------- +--- +SIP +--- SIP is a tool that makes it very easy to create Python bindings for C and C++ libraries. It was originally developed to create PyQt, the Python bindings for @@ -22,7 +22,7 @@ provides support functions to the automatically generated code. Phases ^^^^^^ -The ``SIPPackage`` base class comes with the following phases: +The ``SIPBuilder`` and ``SIPPackage`` base classes come with the following phases: #. ``configure`` - configure the package #. ``build`` - build the package diff --git a/lib/spack/docs/build_systems/wafpackage.rst b/lib/spack/docs/build_systems/wafpackage.rst index 54fcba98d0..f91479ce43 100644 --- a/lib/spack/docs/build_systems/wafpackage.rst +++ b/lib/spack/docs/build_systems/wafpackage.rst @@ -5,9 +5,9 @@ .. _wafpackage: ----------- -WafPackage ----------- +--- +Waf +--- Like SCons, Waf is a general-purpose build system that does not rely on Makefiles to build software. @@ -16,7 +16,7 @@ on Makefiles to build software. Phases ^^^^^^ -The ``WafPackage`` base class comes with the following phases: +The ``WafBuilder`` and ``WafPackage`` base classes come with the following phases: #. ``configure`` - configure the project #. ``build`` - build the project diff --git a/lib/spack/docs/conf.py b/lib/spack/docs/conf.py index 1bdce87238..4fc321c72d 100644 --- a/lib/spack/docs/conf.py +++ b/lib/spack/docs/conf.py @@ -209,6 +209,7 @@ nitpick_ignore = [ # Spack classes that are private and we don't want to expose ("py:class", "spack.provider_index._IndexBase"), ("py:class", "spack.repo._PrependFileLoader"), + ("py:class", "spack.build_systems._checks.BaseBuilder"), # Spack classes that intersphinx is unable to resolve ("py:class", "spack.version.VersionBase"), ] diff --git a/lib/spack/docs/developer_guide.rst b/lib/spack/docs/developer_guide.rst index c4d04cb485..6b67ef9f77 100644 --- a/lib/spack/docs/developer_guide.rst +++ b/lib/spack/docs/developer_guide.rst @@ -149,11 +149,9 @@ grouped by functionality. Package-related modules ^^^^^^^^^^^^^^^^^^^^^^^ -:mod:`spack.package` - Contains the :class:`~spack.package_base.Package` class, which - is the superclass for all packages in Spack. Methods on ``Package`` - implement all phases of the :ref:`package lifecycle - <package-lifecycle>` and manage the build process. +:mod:`spack.package_base` + Contains the :class:`~spack.package_base.PackageBase` class, which + is the superclass for all packages in Spack. :mod:`spack.util.naming` Contains functions for mapping between Spack package names, diff --git a/lib/spack/docs/features.rst b/lib/spack/docs/features.rst index 985da967fd..1682616adb 100644 --- a/lib/spack/docs/features.rst +++ b/lib/spack/docs/features.rst @@ -98,40 +98,42 @@ For example, this command: .. code-block:: console - $ spack create http://www.mr511.de/software/libelf-0.8.13.tar.gz + $ spack create https://ftp.osuosl.org/pub/blfs/conglomeration/libelf/libelf-0.8.13.tar.gz creates a simple python file: .. code-block:: python - from spack import * + from spack.package import * - class Libelf(Package): + class Libelf(AutotoolsPackage): """FIXME: Put a proper description of your package here.""" # FIXME: Add a proper url for your package's homepage here. - homepage = "http://www.example.com" - url = "http://www.mr511.de/software/libelf-0.8.13.tar.gz" + homepage = "https://www.example.com" + url = "https://ftp.osuosl.org/pub/blfs/conglomeration/libelf/libelf-0.8.13.tar.gz" - version('0.8.13', '4136d7b4c04df68b686570afa26988ac') + # FIXME: Add a list of GitHub accounts to + # notify when the package is updated. + # maintainers = ["github_user1", "github_user2"] - # FIXME: Add dependencies if required. - # depends_on('foo') + version("0.8.13", sha256="591a9b4ec81c1f2042a97aa60564e0cb79d041c52faa7416acb38bc95bd2c76d") - def install(self, spec, prefix): - # FIXME: Modify the configure line to suit your build system here. - configure('--prefix={0}'.format(prefix)) + # FIXME: Add dependencies if required. + # depends_on("foo") - # FIXME: Add logic to build and install here. - make() - make('install') + def configure_args(self): + # FIXME: Add arguments other than --prefix + # FIXME: If not needed delete this function + args = [] + return args It doesn't take much python coding to get from there to a working package: .. literalinclude:: _spack_root/var/spack/repos/builtin/packages/libelf/package.py - :lines: 6- + :lines: 5- Spack also provides wrapper functions around common commands like ``configure``, ``make``, and ``cmake`` to make writing packages diff --git a/lib/spack/docs/images/adapter.png b/lib/spack/docs/images/adapter.png Binary files differnew file mode 100644 index 0000000000..aa889c2c32 --- /dev/null +++ b/lib/spack/docs/images/adapter.png diff --git a/lib/spack/docs/images/builder_package_architecture.png b/lib/spack/docs/images/builder_package_architecture.png Binary files differnew file mode 100644 index 0000000000..0c79b1ee7f --- /dev/null +++ b/lib/spack/docs/images/builder_package_architecture.png diff --git a/lib/spack/docs/images/builder_phases.png b/lib/spack/docs/images/builder_phases.png Binary files differnew file mode 100644 index 0000000000..e8141651c4 --- /dev/null +++ b/lib/spack/docs/images/builder_phases.png diff --git a/lib/spack/docs/images/installation_pipeline.png b/lib/spack/docs/images/installation_pipeline.png Binary files differnew file mode 100644 index 0000000000..c1d1e18f1b --- /dev/null +++ b/lib/spack/docs/images/installation_pipeline.png diff --git a/lib/spack/docs/images/original_package_architecture.png b/lib/spack/docs/images/original_package_architecture.png Binary files differnew file mode 100644 index 0000000000..9fc21efcc1 --- /dev/null +++ b/lib/spack/docs/images/original_package_architecture.png diff --git a/lib/spack/docs/images/packaging.excalidrawlib b/lib/spack/docs/images/packaging.excalidrawlib new file mode 100644 index 0000000000..1449c48579 --- /dev/null +++ b/lib/spack/docs/images/packaging.excalidrawlib @@ -0,0 +1,3092 @@ +{ + "type": "excalidrawlib", + "version": 2, + "source": "https://excalidraw.com", + "libraryItems": [ + { + "status": "unpublished", + "elements": [ + { + "type": "rectangle", + "version": 601, + "versionNonce": 158569138, + "isDeleted": false, + "id": "8MYJkzMoNEhDhGH1FB83g", + "fillStyle": "hachure", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 445.75, + "y": 129, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 736, + "height": 651, + "seed": 448140078, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664627195460, + "link": null, + "locked": false + }, + { + "type": "rectangle", + "version": 195, + "versionNonce": 1239338030, + "isDeleted": false, + "id": "2CKbNSYnk0z80hSe6axnR", + "fillStyle": "hachure", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 470.25, + "y": 164, + "strokeColor": "#000000", + "backgroundColor": "#228be6", + "width": 495, + "height": 455, + "seed": 566918834, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "IU_VoaKHNHswI8HaxNWt5", + "type": "arrow" + } + ], + "updated": 1664627105795, + "link": null, + "locked": false + }, + { + "type": "rectangle", + "version": 403, + "versionNonce": 56919410, + "isDeleted": false, + "id": "XUzv2kfpdxMahaSVVS42X", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 509.25, + "y": 407.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 354909550, + "groupIds": [ + "LYqioPcAzrIgJBDV3IaDA", + "SsaCg2uTI9sJjhD323wkh" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "71z_J7hoepiXas8Fk5x0B", + "type": "arrow" + }, + { + "id": "IU_VoaKHNHswI8HaxNWt5", + "type": "arrow" + } + ], + "updated": 1664627099901, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 300, + "versionNonce": 925254318, + "isDeleted": false, + "id": "lkCxvsSEn-AuBHtfj1N0d", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 547.25, + "y": 441, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 321, + "height": 45, + "seed": 1361827954, + "groupIds": [ + "LYqioPcAzrIgJBDV3IaDA", + "SsaCg2uTI9sJjhD323wkh" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664627099902, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "AutotoolsPackage", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "AutotoolsPackage" + }, + { + "type": "rectangle", + "version": 377, + "versionNonce": 1733756722, + "isDeleted": false, + "id": "aCDb2PgRdoFKA8e-GqQzR", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 509.25, + "y": 200, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 175218606, + "groupIds": [ + "WEeFev8dTdo9KgzR3hPki", + "SsaCg2uTI9sJjhD323wkh" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "71z_J7hoepiXas8Fk5x0B", + "type": "arrow" + } + ], + "updated": 1664627099902, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 161, + "versionNonce": 585481454, + "isDeleted": false, + "id": "fXYOlmw0CV0WFTNUDity0", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 627.75, + "y": 233.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 160, + "height": 45, + "seed": 1186724402, + "groupIds": [ + "WEeFev8dTdo9KgzR3hPki", + "SsaCg2uTI9sJjhD323wkh" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664627099902, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "ArpackNg", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "ArpackNg" + }, + { + "type": "arrow", + "version": 290, + "versionNonce": 890458354, + "isDeleted": false, + "id": "71z_J7hoepiXas8Fk5x0B", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 707.8516807799414, + "y": 403, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 0, + "height": 85, + "seed": 247298542, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664627099902, + "link": null, + "locked": false, + "startBinding": { + "focus": 0.02318227093169459, + "gap": 3, + "elementId": "XUzv2kfpdxMahaSVVS42X" + }, + "endBinding": { + "focus": -0.02318227093169459, + "gap": 6, + "elementId": "aCDb2PgRdoFKA8e-GqQzR" + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 0, + -85 + ] + ] + }, + { + "type": "text", + "version": 673, + "versionNonce": 1429991214, + "isDeleted": false, + "id": "bsoYa0EVTdXYsTx5nsFJk", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 783.25, + "y": 518.3821170339361, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 164, + "height": 90, + "seed": 1633805298, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "IU_VoaKHNHswI8HaxNWt5", + "type": "arrow" + } + ], + "updated": 1664627099902, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Package \nHierarchy", + "baseline": 77, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Package \nHierarchy" + }, + { + "type": "rectangle", + "version": 903, + "versionNonce": 1712814318, + "isDeleted": false, + "id": "qRi5xNnAOqg-SFwtYBpoN", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 510.25, + "y": 657.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 1226050606, + "groupIds": [ + "-wCL8N0qNvseDw29hpA8g" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "IU_VoaKHNHswI8HaxNWt5", + "type": "arrow" + } + ], + "updated": 1664627118807, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 623, + "versionNonce": 492299954, + "isDeleted": false, + "id": "9h25d9NB-Q9Wc79boMEnC", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 552.25, + "y": 691, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 313, + "height": 45, + "seed": 186946994, + "groupIds": [ + "-wCL8N0qNvseDw29hpA8g" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664627118807, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Builder Forwarder", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Builder Forwarder" + }, + { + "type": "text", + "version": 1188, + "versionNonce": 351671150, + "isDeleted": false, + "id": "IlomIIocRvEmmYro4MZ68", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1002.75, + "y": 168.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 157, + "height": 90, + "seed": 1428885362, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664627188273, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Package\n Wrapper", + "baseline": 77, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Package\n Wrapper" + }, + { + "type": "arrow", + "version": 832, + "versionNonce": 1121332014, + "isDeleted": false, + "id": "IU_VoaKHNHswI8HaxNWt5", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "dotted", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 707.7778281289579, + "y": 653.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 7.847537838213611, + "height": 130.23576593212783, + "seed": 1301783086, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664627118807, + "link": null, + "locked": false, + "startBinding": { + "elementId": "qRi5xNnAOqg-SFwtYBpoN", + "focus": 0.013062197564634722, + "gap": 4 + }, + "endBinding": { + "elementId": "XUzv2kfpdxMahaSVVS42X", + "focus": 0.056574233332975385, + "gap": 3.7642340678721666 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + -7.847537838213611, + -130.23576593212783 + ] + ] + } + ], + "id": "mulubEO9Lw-HgC00sx7G-", + "created": 1664627205632 + }, + { + "status": "unpublished", + "elements": [ + { + "type": "rectangle", + "version": 360, + "versionNonce": 699609906, + "isDeleted": false, + "id": "ai3MIBTq8Rkokk4d2NJ_k", + "fillStyle": "hachure", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 441.5, + "y": 56, + "strokeColor": "#000000", + "backgroundColor": "#228be6", + "width": 479, + "height": 642, + "seed": 725687342, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926148, + "link": null, + "locked": false + }, + { + "type": "rectangle", + "version": 327, + "versionNonce": 1239118706, + "isDeleted": false, + "id": "7tuXfM91g28UGae9gJkis", + "fillStyle": "hachure", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 993.25, + "y": 53, + "strokeColor": "#000000", + "backgroundColor": "#228be6", + "width": 479, + "height": 642, + "seed": 860539570, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "F6E1EQxM-PyPeNjQXH6NZ", + "type": "arrow" + } + ], + "updated": 1664623054904, + "link": null, + "locked": false + }, + { + "type": "rectangle", + "version": 482, + "versionNonce": 616506034, + "isDeleted": false, + "id": "TmgDkNmbU86sH2Ssf1mL2", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1030.75, + "y": 503.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 329380206, + "groupIds": [ + "rqi4zfKDNJjqgRyIIknBO" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "RQl1RtMzUcPE_zXHt8Ldm", + "type": "arrow" + }, + { + "id": "F6E1EQxM-PyPeNjQXH6NZ", + "type": "arrow" + }, + { + "id": "Iey2r9ev3NqXShFhDRa3t", + "type": "arrow" + } + ], + "updated": 1664623131360, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 377, + "versionNonce": 1649618094, + "isDeleted": false, + "id": "M6LF3AKrGIzDW8p00PLeg", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1068.75, + "y": 537, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 321, + "height": 45, + "seed": 1690477682, + "groupIds": [ + "rqi4zfKDNJjqgRyIIknBO" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926151, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "AutotoolsPackage", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "AutotoolsPackage" + }, + { + "type": "rectangle", + "version": 466, + "versionNonce": 378147058, + "isDeleted": false, + "id": "-34MaUc1fQDbeqLTRUx91", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1030.625, + "y": 296, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 964531118, + "groupIds": [ + "TtAdfrQjw8FIlPZMGmWhX" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "RQl1RtMzUcPE_zXHt8Ldm", + "type": "arrow" + }, + { + "id": "7czUS_PAuM5hdRJoQRDRT", + "type": "arrow" + }, + { + "id": "Iey2r9ev3NqXShFhDRa3t", + "type": "arrow" + } + ], + "updated": 1664623131360, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 250, + "versionNonce": 1826973422, + "isDeleted": false, + "id": "85YHNomCStJoIV17Sp0A6", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1093.625, + "y": 329.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 271, + "height": 45, + "seed": 1436108338, + "groupIds": [ + "TtAdfrQjw8FIlPZMGmWhX" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926151, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "builtin.ArpackNg", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "builtin.ArpackNg" + }, + { + "type": "arrow", + "version": 476, + "versionNonce": 1270564594, + "isDeleted": false, + "id": "RQl1RtMzUcPE_zXHt8Ldm", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1233.8516807799415, + "y": 499, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 0, + "height": 85, + "seed": 1613426158, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926151, + "link": null, + "locked": false, + "startBinding": { + "elementId": "TmgDkNmbU86sH2Ssf1mL2", + "focus": 0.023182270931695163, + "gap": 4.5 + }, + "endBinding": { + "elementId": "-34MaUc1fQDbeqLTRUx91", + "focus": -0.02381199385360952, + "gap": 6 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 0, + -85 + ] + ] + }, + { + "type": "text", + "version": 693, + "versionNonce": 1438013742, + "isDeleted": false, + "id": "wSIdF9zegc69r2D38BVMs", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1061.75, + "y": 632.3821170339361, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 335, + "height": 45, + "seed": 1052094450, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926151, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Old-style packages", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Old-style packages" + }, + { + "type": "rectangle", + "version": 556, + "versionNonce": 1760787058, + "isDeleted": false, + "id": "lYxakYKLpAmo_DvzDJ27b", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1030.625, + "y": 95, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 1302932978, + "groupIds": [ + "-WCCzMWoqGFfWxksMC6LG" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "RQl1RtMzUcPE_zXHt8Ldm", + "type": "arrow" + }, + { + "id": "8Z8HX6DlXqC-qL-63w1ol", + "type": "arrow" + }, + { + "id": "ia8wHuSmOVJLvGe5blR5g", + "type": "arrow" + }, + { + "id": "7czUS_PAuM5hdRJoQRDRT", + "type": "arrow" + } + ], + "updated": 1664623123836, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 341, + "versionNonce": 1412367214, + "isDeleted": false, + "id": "hF1874wuKYmbBjYAQwrVJ", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1088.125, + "y": 128.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 282, + "height": 45, + "seed": 524182062, + "groupIds": [ + "-WCCzMWoqGFfWxksMC6LG" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926152, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "myrepo.ArpackNg", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "myrepo.ArpackNg" + }, + { + "type": "arrow", + "version": 593, + "versionNonce": 214413938, + "isDeleted": false, + "id": "8Z8HX6DlXqC-qL-63w1ol", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "dotted", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1226.4453379157953, + "y": 297.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 2.434529927712447, + "height": 84, + "seed": 1326581486, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926152, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": { + "elementId": "lYxakYKLpAmo_DvzDJ27b", + "focus": -0.00782655608584947, + "gap": 6.5 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 2.434529927712447, + -84 + ] + ] + }, + { + "type": "rectangle", + "version": 733, + "versionNonce": 390297266, + "isDeleted": false, + "id": "G4--cV2YGQSrSijvYiNDB", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 482.5, + "y": 507, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 85080878, + "groupIds": [ + "qZhg7KFANDHKWmTH71Lm0", + "FSKOW2oS76ubMa6DTOrDh" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "RQl1RtMzUcPE_zXHt8Ldm", + "type": "arrow" + }, + { + "id": "BkpnKUCjV1uqDGHPNuNZK", + "type": "arrow" + }, + { + "id": "aQdIO4VQx_J6SzCz-xt64", + "type": "arrow" + }, + { + "id": "F6E1EQxM-PyPeNjQXH6NZ", + "type": "arrow" + } + ], + "updated": 1664623061069, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 577, + "versionNonce": 2001681906, + "isDeleted": false, + "id": "MbNSUrN26Lx1aERuxunnt", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 541.5, + "y": 540.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 279, + "height": 45, + "seed": 950326962, + "groupIds": [ + "qZhg7KFANDHKWmTH71Lm0", + "FSKOW2oS76ubMa6DTOrDh" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926152, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Default Builder", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Default Builder" + }, + { + "type": "rectangle", + "version": 722, + "versionNonce": 1372930162, + "isDeleted": false, + "id": "WIS84sS48dCmi8q81Hh9F", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 482.5, + "y": 99, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 1977410350, + "groupIds": [ + "_CQwHz-xftDZzy8u9u4YO", + "FSKOW2oS76ubMa6DTOrDh" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "RQl1RtMzUcPE_zXHt8Ldm", + "type": "arrow" + }, + { + "id": "BkpnKUCjV1uqDGHPNuNZK", + "type": "arrow" + }, + { + "id": "aQdIO4VQx_J6SzCz-xt64", + "type": "arrow" + }, + { + "id": "ia8wHuSmOVJLvGe5blR5g", + "type": "arrow" + } + ], + "updated": 1664623105535, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 531, + "versionNonce": 1851174834, + "isDeleted": false, + "id": "qIbTXN1LbDYGZzSceYynz", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 540.5, + "y": 132.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 281, + "height": 45, + "seed": 221818546, + "groupIds": [ + "_CQwHz-xftDZzy8u9u4YO", + "FSKOW2oS76ubMa6DTOrDh" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926152, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Adapter Builder", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Adapter Builder" + }, + { + "type": "arrow", + "version": 85, + "versionNonce": 50141422, + "isDeleted": false, + "id": "aQdIO4VQx_J6SzCz-xt64", + "fillStyle": "solid", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "angle": 0, + "x": 670, + "y": 505, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 2, + "height": 291, + "seed": 417372974, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926152, + "link": null, + "locked": false, + "startBinding": { + "elementId": "G4--cV2YGQSrSijvYiNDB", + "focus": -0.05731267980406219, + "gap": 2 + }, + "endBinding": { + "elementId": "WIS84sS48dCmi8q81Hh9F", + "focus": 0.04321344955983103, + "gap": 3 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 2, + -291 + ] + ] + }, + { + "type": "arrow", + "version": 720, + "versionNonce": 1494556718, + "isDeleted": false, + "id": "ia8wHuSmOVJLvGe5blR5g", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 4.706831282597808, + "x": 932.4285606227004, + "y": 52.69401049592016, + "strokeColor": "#c92a2a", + "backgroundColor": "transparent", + "width": 47.10077935537049, + "height": 145.9883132350331, + "seed": 314146734, + "groupIds": [], + "strokeSharpness": "round", + "boundElements": [], + "updated": 1664623039605, + "link": null, + "locked": false, + "startBinding": { + "elementId": "WIS84sS48dCmi8q81Hh9F", + "focus": 0.6597923311816741, + "gap": 2.0985583595166872 + }, + "endBinding": { + "elementId": "lYxakYKLpAmo_DvzDJ27b", + "focus": -0.6857137990945498, + "gap": 3.0336827015810286 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 45.89517648378751, + 72.7218231059162 + ], + [ + -1.2056028715829825, + 145.9883132350331 + ] + ] + }, + { + "type": "text", + "version": 727, + "versionNonce": 549636846, + "isDeleted": false, + "id": "JRrvIVZ9KAv56BYbRbCLA", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 527.5, + "y": 633.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 295, + "height": 45, + "seed": 2130028978, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664622926153, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Builder Hierarchy", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Builder Hierarchy" + }, + { + "type": "text", + "version": 281, + "versionNonce": 777063918, + "isDeleted": false, + "id": "BBj29IYUUwcEAk0aGGgEe", + "fillStyle": "solid", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "angle": 0, + "x": 746, + "y": 2, + "strokeColor": "#c92a2a", + "backgroundColor": "#ced4da", + "width": 438, + "height": 35, + "seed": 344107566, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664623034966, + "link": null, + "locked": false, + "fontSize": 28, + "fontFamily": 1, + "text": "Defer to the old-style package", + "baseline": 25, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Defer to the old-style package" + }, + { + "type": "arrow", + "version": 864, + "versionNonce": 353999662, + "isDeleted": false, + "id": "F6E1EQxM-PyPeNjQXH6NZ", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 1.5656833824867196, + "x": 932.5276780900645, + "y": 511.2079252998286, + "strokeColor": "#c92a2a", + "backgroundColor": "transparent", + "width": 47.10077935537049, + "height": 145.9883132350331, + "seed": 2119154546, + "groupIds": [], + "strokeSharpness": "round", + "boundElements": [], + "updated": 1664623061069, + "link": null, + "locked": false, + "startBinding": { + "elementId": "TmgDkNmbU86sH2Ssf1mL2", + "focus": 0.700636908798286, + "gap": 3.7338363313426726 + }, + "endBinding": { + "elementId": "G4--cV2YGQSrSijvYiNDB", + "focus": -0.7137516210459195, + "gap": 1.5235945037890133 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 45.89517648378751, + 72.7218231059162 + ], + [ + -1.2056028715829825, + 145.9883132350331 + ] + ] + }, + { + "type": "text", + "version": 318, + "versionNonce": 1988243186, + "isDeleted": false, + "id": "VIOq-st9nvReenpiJkr7q", + "fillStyle": "solid", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "angle": 0, + "x": 828, + "y": 724.5, + "strokeColor": "#c92a2a", + "backgroundColor": "#ced4da", + "width": 274, + "height": 70, + "seed": 2086072882, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664623095297, + "link": null, + "locked": false, + "fontSize": 28, + "fontFamily": 1, + "text": "Fall-back to the \nAdapter base class", + "baseline": 60, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Fall-back to the \nAdapter base class" + }, + { + "type": "arrow", + "version": 971, + "versionNonce": 1844256174, + "isDeleted": false, + "id": "7czUS_PAuM5hdRJoQRDRT", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 6.272294617229998, + "x": 1433.5276780900645, + "y": 163.20792529982862, + "strokeColor": "#c92a2a", + "backgroundColor": "transparent", + "width": 47.10077935537049, + "height": 145.9883132350331, + "seed": 142056302, + "groupIds": [], + "strokeSharpness": "round", + "boundElements": [], + "updated": 1664623123836, + "link": null, + "locked": false, + "startBinding": { + "elementId": "lYxakYKLpAmo_DvzDJ27b", + "focus": -0.8331982906950285, + "gap": 5.098981289624589 + }, + "endBinding": { + "elementId": "-34MaUc1fQDbeqLTRUx91", + "focus": 0.7587321286266477, + "gap": 5.483331940596372 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 45.89517648378751, + 72.7218231059162 + ], + [ + -1.2056028715829825, + 145.9883132350331 + ] + ] + }, + { + "type": "arrow", + "version": 1075, + "versionNonce": 2073112366, + "isDeleted": false, + "id": "Iey2r9ev3NqXShFhDRa3t", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 6.272294617229998, + "x": 1434.451400933309, + "y": 387.7559332541056, + "strokeColor": "#c92a2a", + "backgroundColor": "transparent", + "width": 47.10077935537049, + "height": 145.9883132350331, + "seed": 840513518, + "groupIds": [], + "strokeSharpness": "round", + "boundElements": [], + "updated": 1664623131360, + "link": null, + "locked": false, + "startBinding": { + "elementId": "-34MaUc1fQDbeqLTRUx91", + "focus": -0.7723329153292293, + "gap": 6.037577244264867 + }, + "endBinding": { + "elementId": "TmgDkNmbU86sH2Ssf1mL2", + "focus": 0.808011962769455, + "gap": 6.296927895236422 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 45.89517648378751, + 72.7218231059162 + ], + [ + -1.2056028715829825, + 145.9883132350331 + ] + ] + } + ], + "id": "sJP5ES4-kuhrqaBed7Feh", + "created": 1664623142493 + }, + { + "status": "unpublished", + "elements": [ + { + "type": "rectangle", + "version": 351, + "versionNonce": 94847218, + "isDeleted": false, + "id": "QfhQQY4Kvx8RLvCd6qXsx", + "fillStyle": "hachure", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1011.5, + "y": 249, + "strokeColor": "#000000", + "backgroundColor": "#228be6", + "width": 479, + "height": 438, + "seed": 1024685106, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664612347442, + "link": null, + "locked": false + }, + { + "type": "rectangle", + "version": 156, + "versionNonce": 2082406190, + "isDeleted": false, + "id": "rMQqqzkSZsBVWvOk137wO", + "fillStyle": "hachure", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 511, + "y": 247, + "strokeColor": "#000000", + "backgroundColor": "#228be6", + "width": 479, + "height": 438, + "seed": 250617778, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664612347443, + "link": null, + "locked": false + }, + { + "type": "rectangle", + "version": 392, + "versionNonce": 414601906, + "isDeleted": false, + "id": "h2lcAgJBn6WsPKAj3vWS8", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 545.5, + "y": 490.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 721668433, + "groupIds": [ + "ETPwHpdW1CXh0DtqZ_2na" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "r2Lq0kGXd6aTn5T-ki1aL", + "type": "arrow" + } + ], + "updated": 1664612347443, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 293, + "versionNonce": 848488814, + "isDeleted": false, + "id": "eaxk_MzyrjAjXKf0vmFuU", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 583.5, + "y": 524, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 321, + "height": 45, + "seed": 1324675135, + "groupIds": [ + "ETPwHpdW1CXh0DtqZ_2na" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664612347443, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "AutotoolsPackage", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "AutotoolsPackage" + }, + { + "type": "rectangle", + "version": 370, + "versionNonce": 595405938, + "isDeleted": false, + "id": "6TAhmS7GKN_ppUHjSVGLb", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 545.5, + "y": 283, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 2083634783, + "groupIds": [ + "biKtN87UToAb_UBhyub5I" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "r2Lq0kGXd6aTn5T-ki1aL", + "type": "arrow" + } + ], + "updated": 1664612347443, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 155, + "versionNonce": 1066372014, + "isDeleted": false, + "id": "xyXchzGRLKRPuMVGo17mr", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 664, + "y": 316.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 160, + "height": 45, + "seed": 2066951921, + "groupIds": [ + "biKtN87UToAb_UBhyub5I" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664612347443, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "ArpackNg", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "ArpackNg" + }, + { + "type": "arrow", + "version": 285, + "versionNonce": 1807928882, + "isDeleted": false, + "id": "r2Lq0kGXd6aTn5T-ki1aL", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 748.6016807799414, + "y": 486, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 0, + "height": 85, + "seed": 1479060383, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664612347443, + "link": null, + "locked": false, + "startBinding": { + "elementId": "h2lcAgJBn6WsPKAj3vWS8", + "focus": 0.02318227093169459, + "gap": 3 + }, + "endBinding": { + "elementId": "6TAhmS7GKN_ppUHjSVGLb", + "focus": -0.02318227093169459, + "gap": 6 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 0, + -85 + ] + ] + }, + { + "type": "text", + "version": 572, + "versionNonce": 1094575598, + "isDeleted": false, + "id": "pUx1_v_UyKhu5zXISU4-f", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 653, + "y": 619.3821170339361, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 182, + "height": 45, + "seed": 1608256017, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664612347443, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Metadata", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Metadata" + }, + { + "type": "rectangle", + "version": 734, + "versionNonce": 1401317810, + "isDeleted": false, + "id": "4YBPHTc5sQiOKGM9NOZwg", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1045.5, + "y": 490.5, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 396.99999999999994, + "height": 112, + "seed": 1687989426, + "groupIds": [ + "lxE4hLtUAF2X7993lwk8q" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "M8cWqpsa0-iwN_cVJeXEQ", + "type": "arrow" + } + ], + "updated": 1664612347443, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 436, + "versionNonce": 1572061806, + "isDeleted": false, + "id": "P2U0ucf_QPvJcOWlMLp2K", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1183, + "y": 524, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 122, + "height": 45, + "seed": 276038958, + "groupIds": [ + "lxE4hLtUAF2X7993lwk8q" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664612347443, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Builder", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Builder" + }, + { + "type": "arrow", + "version": 489, + "versionNonce": 1663911086, + "isDeleted": false, + "id": "M8cWqpsa0-iwN_cVJeXEQ", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "dashed", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 942, + "y": 337, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 303, + "height": 143, + "seed": 1698960686, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664612347443, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": { + "elementId": "4YBPHTc5sQiOKGM9NOZwg", + "focus": 0.04820781382766574, + "gap": 10.5 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "dot", + "points": [ + [ + 0, + 0 + ], + [ + 295, + 0 + ], + [ + 303, + 143 + ] + ] + }, + { + "type": "text", + "version": 841, + "versionNonce": 2059173614, + "isDeleted": false, + "id": "QGyg9pXnTgByg9Lw9oZKC", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1043.5, + "y": 621.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 401, + "height": 45, + "seed": 1012078510, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664612347443, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Installation Procedure", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Installation Procedure" + } + ], + "id": "tezI4Q4gBH7mr-Q_us1KO", + "created": 1664612353293 + }, + { + "status": "unpublished", + "elements": [ + { + "type": "rectangle", + "version": 273, + "versionNonce": 1078330865, + "isDeleted": false, + "id": "h2lcAgJBn6WsPKAj3vWS8", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 545.5, + "y": 489, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 396.99999999999994, + "height": 112, + "seed": 721668433, + "groupIds": [ + "ETPwHpdW1CXh0DtqZ_2na" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "r2Lq0kGXd6aTn5T-ki1aL", + "type": "arrow" + } + ], + "updated": 1664534889868, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 174, + "versionNonce": 1400524191, + "isDeleted": false, + "id": "eaxk_MzyrjAjXKf0vmFuU", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 583.5, + "y": 522.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 321, + "height": 45, + "seed": 1324675135, + "groupIds": [ + "ETPwHpdW1CXh0DtqZ_2na" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664534889868, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "AutotoolsPackage", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "AutotoolsPackage" + }, + { + "type": "text", + "version": 108, + "versionNonce": 438728849, + "isDeleted": false, + "id": "xyXchzGRLKRPuMVGo17mr", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 664, + "y": 316.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 160, + "height": 45, + "seed": 2066951921, + "groupIds": [ + "1wm7ikIN28k9zdVSKTLKQ" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664540120970, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "ArpackNg", + "baseline": 32, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "ArpackNg" + }, + { + "type": "rectangle", + "version": 322, + "versionNonce": 1389146591, + "isDeleted": false, + "id": "6TAhmS7GKN_ppUHjSVGLb", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 545.5, + "y": 283, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 396.99999999999994, + "height": 112, + "seed": 2083634783, + "groupIds": [ + "1wm7ikIN28k9zdVSKTLKQ" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "r2Lq0kGXd6aTn5T-ki1aL", + "type": "arrow" + } + ], + "updated": 1664534889868, + "link": null, + "locked": false + }, + { + "type": "arrow", + "version": 94, + "versionNonce": 787416433, + "isDeleted": false, + "id": "r2Lq0kGXd6aTn5T-ki1aL", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 748.6016807799414, + "y": 486, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 0, + "height": 85, + "seed": 1479060383, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664534889868, + "link": null, + "locked": false, + "startBinding": { + "elementId": "h2lcAgJBn6WsPKAj3vWS8", + "focus": 0.02318227093169459, + "gap": 3 + }, + "endBinding": { + "elementId": "6TAhmS7GKN_ppUHjSVGLb", + "focus": -0.02318227093169459, + "gap": 6 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 0, + -85 + ] + ] + }, + { + "type": "text", + "version": 227, + "versionNonce": 117980031, + "isDeleted": false, + "id": "pUx1_v_UyKhu5zXISU4-f", + "fillStyle": "hachure", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 969, + "y": 386.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 442, + "height": 90, + "seed": 1608256017, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1664534908931, + "link": null, + "locked": false, + "fontSize": 36, + "fontFamily": 1, + "text": "Metadata \n+ Installation Procedure", + "baseline": 77, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "Metadata \n+ Installation Procedure" + } + ], + "id": "_c7AOn60omrTlppZHlLQh", + "created": 1664540190548 + }, + { + "status": "unpublished", + "elements": [ + { + "type": "rectangle", + "version": 367, + "versionNonce": 963584621, + "isDeleted": false, + "id": "oAei2n-Ha1gpjnYdK7AwC", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 240.5, + "y": 642.75, + "strokeColor": "#000000", + "backgroundColor": "#228be6", + "width": 392, + "height": 80, + "seed": 701868237, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "slfbd0bbRqA8648kZ5fns", + "type": "text" + }, + { + "id": "slfbd0bbRqA8648kZ5fns", + "type": "text" + }, + { + "type": "text", + "id": "slfbd0bbRqA8648kZ5fns" + } + ], + "updated": 1663329462351, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 373, + "versionNonce": 1698441027, + "isDeleted": false, + "id": "slfbd0bbRqA8648kZ5fns", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 245.5, + "y": 670.25, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 382, + "height": 25, + "seed": 1179637379, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1663329462351, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "Execute the installation process", + "baseline": 18, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "oAei2n-Ha1gpjnYdK7AwC", + "originalText": "Execute the installation process" + }, + { + "type": "rectangle", + "version": 208, + "versionNonce": 844908259, + "isDeleted": false, + "id": "cLwg2WXUit_OTQmXLIdIW", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 815.5, + "y": 517.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 392, + "height": 80, + "seed": 557411811, + "groupIds": [ + "D1SCf714tngJFHk8TFX8T" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "SpG_8HxzMHjM2HYK6Fgwx", + "type": "arrow" + } + ], + "updated": 1663329462351, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 274, + "versionNonce": 1704611021, + "isDeleted": false, + "id": "1r8FMl26VYSKpPKlHA_Oc", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 916.5, + "y": 545, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 207, + "height": 25, + "seed": 961881101, + "groupIds": [ + "D1SCf714tngJFHk8TFX8T" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1663329462351, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "CMakeBuilder.cmake()", + "baseline": 18, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "CMakeBuilder.cmake()" + }, + { + "type": "rectangle", + "version": 264, + "versionNonce": 295137923, + "isDeleted": false, + "id": "CSwjuAw6Nl67sqQ6p21ty", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 815.5, + "y": 642.75, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 392, + "height": 80, + "seed": 1011629069, + "groupIds": [ + "D1SCf714tngJFHk8TFX8T" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "SpG_8HxzMHjM2HYK6Fgwx", + "type": "arrow" + }, + { + "id": "zvmLoAH5oICRD5og-pBvu", + "type": "arrow" + } + ], + "updated": 1663329462351, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 466, + "versionNonce": 196160301, + "isDeleted": false, + "id": "WX4axTU0IR7PJb0GkR-jq", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 922.5, + "y": 670.25, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 193, + "height": 25, + "seed": 716117827, + "groupIds": [ + "D1SCf714tngJFHk8TFX8T" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1663329462351, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "CMakeBuilder.build()", + "baseline": 18, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "CMakeBuilder.build()" + }, + { + "type": "rectangle", + "version": 301, + "versionNonce": 1545420173, + "isDeleted": false, + "id": "coUXke3Fv_DpjqG9zgEjQ", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 815.5, + "y": 768, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 392, + "height": 80, + "seed": 1934529891, + "groupIds": [ + "D1SCf714tngJFHk8TFX8T" + ], + "strokeSharpness": "sharp", + "boundElements": [ + { + "type": "text", + "id": "yVIbU03yFYvpXnh9xIgET" + }, + { + "id": "zvmLoAH5oICRD5og-pBvu", + "type": "arrow" + } + ], + "updated": 1663329462351, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 273, + "versionNonce": 1837690307, + "isDeleted": false, + "id": "yVIbU03yFYvpXnh9xIgET", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 820.5, + "y": 795.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 382, + "height": 25, + "seed": 1611291683, + "groupIds": [ + "D1SCf714tngJFHk8TFX8T" + ], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1663329462351, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "CMakeBuilder.install()", + "baseline": 18, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "coUXke3Fv_DpjqG9zgEjQ", + "originalText": "CMakeBuilder.install()" + }, + { + "type": "arrow", + "version": 564, + "versionNonce": 1041761261, + "isDeleted": false, + "id": "SpG_8HxzMHjM2HYK6Fgwx", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1209, + "y": 558.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 96, + "height": 109, + "seed": 732445197, + "groupIds": [ + "D1SCf714tngJFHk8TFX8T" + ], + "strokeSharpness": "round", + "boundElements": [], + "updated": 1663329462351, + "link": null, + "locked": false, + "startBinding": { + "elementId": "cLwg2WXUit_OTQmXLIdIW", + "focus": -0.7327371048252911, + "gap": 1.5 + }, + "endBinding": { + "elementId": "CSwjuAw6Nl67sqQ6p21ty", + "focus": 0.6494341563786008, + "gap": 2.5 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 96, + 54 + ], + [ + 1, + 109 + ] + ] + }, + { + "type": "arrow", + "version": 642, + "versionNonce": 1380728163, + "isDeleted": false, + "id": "zvmLoAH5oICRD5og-pBvu", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1216, + "y": 680, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 98, + "height": 124.33745608356844, + "seed": 708861581, + "groupIds": [ + "D1SCf714tngJFHk8TFX8T" + ], + "strokeSharpness": "round", + "boundElements": [], + "updated": 1663329462351, + "link": null, + "locked": false, + "startBinding": { + "elementId": "CSwjuAw6Nl67sqQ6p21ty", + "focus": -0.7839018302828619, + "gap": 8.5 + }, + "endBinding": { + "elementId": "coUXke3Fv_DpjqG9zgEjQ", + "focus": 0.7841576120638036, + "gap": 6.5 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 96, + 54 + ], + [ + -2, + 124.33745608356844 + ] + ] + }, + { + "type": "text", + "version": 613, + "versionNonce": 909390253, + "isDeleted": false, + "id": "fAHH1YdSlMq8ioLIj36Of", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 644, + "y": 353.7484662576685, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 166, + "height": 567.2515337423315, + "seed": 1455993539, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1663329499644, + "link": null, + "locked": false, + "fontSize": 493.16790307261914, + "fontFamily": 2, + "text": "{", + "baseline": 454.2515337423315, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "{" + } + ], + "id": "KBV_I9pxrJD2zPuaP6vBc", + "created": 1663329511286 + }, + { + "status": "unpublished", + "elements": [ + { + "type": "rectangle", + "version": 93, + "versionNonce": 42296109, + "isDeleted": false, + "id": "cLwg2WXUit_OTQmXLIdIW", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 625.5, + "y": 298, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 392, + "height": 80, + "seed": 557411811, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "SpG_8HxzMHjM2HYK6Fgwx", + "type": "arrow" + } + ], + "updated": 1663324636434, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 99, + "versionNonce": 1537897869, + "isDeleted": false, + "id": "1r8FMl26VYSKpPKlHA_Oc", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 726.5, + "y": 325.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 179, + "height": 25, + "seed": 961881101, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1663324636434, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "Fetch source files", + "baseline": 18, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Fetch source files" + }, + { + "type": "rectangle", + "version": 149, + "versionNonce": 1653290435, + "isDeleted": false, + "id": "CSwjuAw6Nl67sqQ6p21ty", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 625.5, + "y": 423.25, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 392, + "height": 80, + "seed": 1011629069, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "SpG_8HxzMHjM2HYK6Fgwx", + "type": "arrow" + }, + { + "id": "zvmLoAH5oICRD5og-pBvu", + "type": "arrow" + } + ], + "updated": 1663324636434, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 272, + "versionNonce": 1195260909, + "isDeleted": false, + "id": "WX4axTU0IR7PJb0GkR-jq", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 645.5, + "y": 450.75, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 352, + "height": 25, + "seed": 716117827, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1663324636434, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "Expand them in the stage directory", + "baseline": 18, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Expand them in the stage directory" + }, + { + "type": "rectangle", + "version": 185, + "versionNonce": 2143651171, + "isDeleted": false, + "id": "coUXke3Fv_DpjqG9zgEjQ", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 625.5, + "y": 548.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 392, + "height": 80, + "seed": 1934529891, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [ + { + "type": "text", + "id": "yVIbU03yFYvpXnh9xIgET" + }, + { + "id": "zvmLoAH5oICRD5og-pBvu", + "type": "arrow" + }, + { + "id": "5yqrFWV-hhJ4RoVewqAC0", + "type": "arrow" + } + ], + "updated": 1663324636434, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 135, + "versionNonce": 1833580109, + "isDeleted": false, + "id": "yVIbU03yFYvpXnh9xIgET", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 630.5, + "y": 563.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 382, + "height": 50, + "seed": 1611291683, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1663324636434, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "Set the stage directory as the \ncurrent working directory", + "baseline": 43, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "coUXke3Fv_DpjqG9zgEjQ", + "originalText": "Set the stage directory as the current working directory" + }, + { + "type": "rectangle", + "version": 253, + "versionNonce": 1704770627, + "isDeleted": false, + "id": "tBTBRiEA6AJABK4wnKF_-", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 625.5, + "y": 673.75, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 392, + "height": 80, + "seed": 1257829773, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "GoE9udjDQxUqdsYCUnbVI", + "type": "text" + }, + { + "type": "text", + "id": "GoE9udjDQxUqdsYCUnbVI" + }, + { + "id": "5yqrFWV-hhJ4RoVewqAC0", + "type": "arrow" + }, + { + "id": "v-9Voh5erXQ8iqoQ_9BVO", + "type": "arrow" + } + ], + "updated": 1663324636434, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 194, + "versionNonce": 1557028205, + "isDeleted": false, + "id": "GoE9udjDQxUqdsYCUnbVI", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 630.5, + "y": 701.25, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 382, + "height": 25, + "seed": 895792579, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1663324636434, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "Fork a new build environment", + "baseline": 18, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "tBTBRiEA6AJABK4wnKF_-", + "originalText": "Fork a new build environment" + }, + { + "type": "rectangle", + "version": 321, + "versionNonce": 1675770851, + "isDeleted": false, + "id": "oAei2n-Ha1gpjnYdK7AwC", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 625.5, + "y": 799, + "strokeColor": "#000000", + "backgroundColor": "#228be6", + "width": 392, + "height": 80, + "seed": 701868237, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [ + { + "id": "slfbd0bbRqA8648kZ5fns", + "type": "text" + }, + { + "id": "slfbd0bbRqA8648kZ5fns", + "type": "text" + }, + { + "type": "text", + "id": "slfbd0bbRqA8648kZ5fns" + }, + { + "id": "v-9Voh5erXQ8iqoQ_9BVO", + "type": "arrow" + } + ], + "updated": 1663324636434, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 328, + "versionNonce": 1868179405, + "isDeleted": false, + "id": "slfbd0bbRqA8648kZ5fns", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 630.5, + "y": 826.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 382, + "height": 25, + "seed": 1179637379, + "groupIds": [], + "strokeSharpness": "sharp", + "boundElements": [], + "updated": 1663324636434, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "Execute the installation process", + "baseline": 18, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "oAei2n-Ha1gpjnYdK7AwC", + "originalText": "Execute the installation process" + }, + { + "type": "arrow", + "version": 221, + "versionNonce": 1777917731, + "isDeleted": false, + "id": "SpG_8HxzMHjM2HYK6Fgwx", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1019, + "y": 339, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 96, + "height": 109, + "seed": 732445197, + "groupIds": [], + "strokeSharpness": "round", + "boundElements": [], + "updated": 1663324636434, + "link": null, + "locked": false, + "startBinding": { + "elementId": "cLwg2WXUit_OTQmXLIdIW", + "focus": -0.7533277870216306, + "gap": 7 + }, + "endBinding": { + "elementId": "CSwjuAw6Nl67sqQ6p21ty", + "focus": 0.7554869684499315, + "gap": 6 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 96, + 54 + ], + [ + 1, + 109 + ] + ] + }, + { + "type": "arrow", + "version": 299, + "versionNonce": 309789379, + "isDeleted": false, + "id": "zvmLoAH5oICRD5og-pBvu", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1026, + "y": 460.5, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 98, + "height": 124.33745608356844, + "seed": 708861581, + "groupIds": [], + "strokeSharpness": "round", + "boundElements": [], + "updated": 1663324636435, + "link": null, + "locked": false, + "startBinding": { + "elementId": "CSwjuAw6Nl67sqQ6p21ty", + "focus": -0.7021630615640598, + "gap": 12 + }, + "endBinding": { + "elementId": "coUXke3Fv_DpjqG9zgEjQ", + "focus": 0.8530521262002744, + "gap": 12 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 96, + 54 + ], + [ + -2, + 124.33745608356844 + ] + ] + }, + { + "type": "arrow", + "version": 301, + "versionNonce": 914472685, + "isDeleted": false, + "id": "5yqrFWV-hhJ4RoVewqAC0", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1019, + "y": 586.6789496258876, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 99, + "height": 123.78306157234579, + "seed": 642378381, + "groupIds": [], + "strokeSharpness": "round", + "boundElements": [], + "updated": 1663324636435, + "link": null, + "locked": false, + "startBinding": { + "elementId": "coUXke3Fv_DpjqG9zgEjQ", + "focus": -0.6501663893510815, + "gap": 7 + }, + "endBinding": { + "elementId": "tBTBRiEA6AJABK4wnKF_-", + "focus": 0.8705418381344308, + "gap": 8 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 99, + 42.82105037411236 + ], + [ + 10.000000000000227, + 123.78306157234579 + ] + ] + }, + { + "type": "arrow", + "version": 351, + "versionNonce": 984592995, + "isDeleted": false, + "id": "v-9Voh5erXQ8iqoQ_9BVO", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 2, + "opacity": 100, + "angle": 0, + "x": 1031, + "y": 714.8662394200408, + "strokeColor": "#000000", + "backgroundColor": "transparent", + "width": 90, + "height": 137.7637151210173, + "seed": 698547757, + "groupIds": [], + "strokeSharpness": "round", + "boundElements": [], + "updated": 1663324636435, + "link": null, + "locked": false, + "startBinding": { + "elementId": "tBTBRiEA6AJABK4wnKF_-", + "focus": -0.6014975041597337, + "gap": 10 + }, + "endBinding": { + "elementId": "oAei2n-Ha1gpjnYdK7AwC", + "focus": 0.9573045267489712, + "gap": 12 + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 90, + 33.633760579959244 + ], + [ + 4, + 137.7637151210173 + ] + ] + } + ], + "id": "RzNgncGu1938Ma5Teh6qZ", + "created": 1663324659550 + } + ] +}
\ No newline at end of file diff --git a/lib/spack/docs/packaging_guide.rst b/lib/spack/docs/packaging_guide.rst index 556dde31e8..f7b8cfe35f 100644 --- a/lib/spack/docs/packaging_guide.rst +++ b/lib/spack/docs/packaging_guide.rst @@ -34,24 +34,155 @@ ubiquitous in the scientific software community. Second, it's a modern language and has many powerful features to help make package writing easy. ---------------------------- -Creating & editing packages ---------------------------- -.. _cmd-spack-create: +.. _installation_procedure: -^^^^^^^^^^^^^^^^ -``spack create`` -^^^^^^^^^^^^^^^^ +-------------------------------------- +Overview of the installation procedure +-------------------------------------- + +Whenever Spack installs software, it goes through a series of predefined steps: -The ``spack create`` command creates a directory with the package name and -generates a ``package.py`` file with a boilerplate package template. If given -a URL pointing to a tarball or other software archive, ``spack create`` is -smart enough to determine basic information about the package, including its name -and build system. In most cases, ``spack create`` plus a few modifications is -all you need to get a package working. +.. image:: images/installation_pipeline.png + :scale: 60 % + :align: center -Here's an example: +All these steps are influenced by the metadata in each ``package.py`` and +by the current Spack configuration. +Since build systems are different from one another, the execution of the +last block in the figure is further expanded in a build system specific way. +An example for ``CMake`` is, for instance: + +.. image:: images/builder_phases.png + :align: center + :scale: 60 % + +The predefined steps for each build system are called "phases". +In general, the name and order in which the phases will be executed can be +obtained by either reading the API docs at :py:mod:`~.spack.build_systems`, or +using the ``spack info`` command: + +.. code-block:: console + :emphasize-lines: 13,14 + + $ spack info --phases m4 + AutotoolsPackage: m4 + Homepage: https://www.gnu.org/software/m4/m4.html + + Safe versions: + 1.4.17 ftp://ftp.gnu.org/gnu/m4/m4-1.4.17.tar.gz + + Variants: + Name Default Description + + sigsegv on Build the libsigsegv dependency + + Installation Phases: + autoreconf configure build install + + Build Dependencies: + libsigsegv + + ... + +An extensive list of available build systems and phases is provided in :ref:`installation_process`. + + +------------------------ +Writing a package recipe +------------------------ + +Since v0.19, Spack supports two ways of writing a package recipe. The most commonly used is to encode both the metadata +(directives, etc.) and the build behavior in a single class, like shown in the following example: + +.. code-block:: python + + class Openjpeg(CMakePackage): + """OpenJPEG is an open-source JPEG 2000 codec written in C language""" + + homepage = "https://github.com/uclouvain/openjpeg" + url = "https://github.com/uclouvain/openjpeg/archive/v2.3.1.tar.gz" + + version("2.4.0", sha256="8702ba68b442657f11aaeb2b338443ca8d5fb95b0d845757968a7be31ef7f16d") + + variant("codec", default=False, description="Build the CODEC executables") + depends_on("libpng", when="+codec") + + def url_for_version(self, version): + if version >= Version("2.1.1"): + return super(Openjpeg, self).url_for_version(version) + url_fmt = "https://github.com/uclouvain/openjpeg/archive/version.{0}.tar.gz" + return url_fmt.format(version) + + def cmake_args(self): + args = [ + self.define_from_variant("BUILD_CODEC", "codec"), + self.define("BUILD_MJ2", False), + self.define("BUILD_THIRDPARTY", False), + ] + return args + +A package encoded with a single class is backward compatible with versions of Spack +lower than v0.19, and so are custom repositories containing only recipes of this kind. +The downside is that *this format doesn't allow packagers to use more than one build system in a single recipe*. + +To do that, we have to resort to the second way Spack has of writing packages, which involves writing a +builder class explicitly. Using the same example as above, this reads: + +.. code-block:: python + + class Openjpeg(CMakePackage): + """OpenJPEG is an open-source JPEG 2000 codec written in C language""" + + homepage = "https://github.com/uclouvain/openjpeg" + url = "https://github.com/uclouvain/openjpeg/archive/v2.3.1.tar.gz" + + version("2.4.0", sha256="8702ba68b442657f11aaeb2b338443ca8d5fb95b0d845757968a7be31ef7f16d") + + variant("codec", default=False, description="Build the CODEC executables") + depends_on("libpng", when="+codec") + + def url_for_version(self, version): + if version >= Version("2.1.1"): + return super(Openjpeg, self).url_for_version(version) + url_fmt = "https://github.com/uclouvain/openjpeg/archive/version.{0}.tar.gz" + return url_fmt.format(version) + + class CMakeBuilder(spack.build_systems.cmake.CMakeBuilder): + def cmake_args(self): + args = [ + self.define_from_variant("BUILD_CODEC", "codec"), + self.define("BUILD_MJ2", False), + self.define("BUILD_THIRDPARTY", False), + ] + return args + +This way of writing packages allows extending the recipe to support multiple build systems, +see :ref:`multiple_build_systems` for more details. The downside is that recipes of this kind +are only understood by Spack since v0.19+. More information on the internal architecture of +Spack can be found at :ref:`package_class_structure`. + +.. note:: + + If a builder is implemented in ``package.py``, all build-specific methods must be moved + to the builder. This means that if you have a package like + + .. code-block:: python + + class Foo(CmakePackage): + def cmake_args(self): + ... + + and you add a builder to the ``package.py``, you must move ``cmake_args`` to the builder. + +.. _cmd-spack-create: + +--------------------- +Creating new packages +--------------------- + +To help creating a new package Spack provides a command that generates a ``package.py`` +file in an existing repository, with a boilerplate package template. Here's an example: .. code-block:: console @@ -87,23 +218,6 @@ You do not *have* to download all of the versions up front. You can always choose to download just one tarball initially, and run :ref:`cmd-spack-checksum` later if you need more versions. -Let's say you download 3 tarballs: - -.. code-block:: console - - How many would you like to checksum? (default is 1, q to abort) 3 - ==> Downloading... - ==> Fetching https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2 - ######################################################################## 100.0% - ==> Fetching https://gmplib.org/download/gmp/gmp-6.1.1.tar.bz2 - ######################################################################## 100.0% - ==> Fetching https://gmplib.org/download/gmp/gmp-6.1.0.tar.bz2 - ######################################################################## 100.0% - ==> Checksummed 3 versions of gmp: - ==> This package looks like it uses the autotools build system - ==> Created template for gmp package - ==> Created package file: /Users/Adam/spack/var/spack/repos/builtin/packages/gmp/package.py - Spack automatically creates a directory in the appropriate repository, generates a boilerplate template for your package, and opens up the new ``package.py`` in your favorite ``$EDITOR``: @@ -111,6 +225,14 @@ generates a boilerplate template for your package, and opens up the new .. code-block:: python :linenos: + # Copyright 2013-2022 Lawrence Livermore National Security, LLC and other + # Spack Project Developers. See the top-level COPYRIGHT file for details. + # + # SPDX-License-Identifier: (Apache-2.0 OR MIT) + + # ---------------------------------------------------------------------------- + # If you submit this package back to Spack as a pull request, + # please first remove this boilerplate and all FIXME comments. # # This is a template package file for Spack. We've put "FIXME" # next to all the things you'll want to change. Once you've handled @@ -123,9 +245,8 @@ generates a boilerplate template for your package, and opens up the new # spack edit gmp # # See the Spack documentation for more information on packaging. - # If you submit this package back to Spack as a pull request, - # please first remove this boilerplate and all FIXME comments. - # + # ---------------------------------------------------------------------------- + import spack.build_systems.autotools from spack.package import * @@ -133,19 +254,17 @@ generates a boilerplate template for your package, and opens up the new """FIXME: Put a proper description of your package here.""" # FIXME: Add a proper url for your package's homepage here. - homepage = "http://www.example.com" - url = "https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2" + homepage = "https://www.example.com" + url = "https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2" # FIXME: Add a list of GitHub accounts to # notify when the package is updated. - # maintainers = ['github_user1', 'github_user2'] + # maintainers = ["github_user1", "github_user2"] - version('6.1.2', '8ddbb26dc3bd4e2302984debba1406a5') - version('6.1.1', '4c175f86e11eb32d8bf9872ca3a8e11d') - version('6.1.0', '86ee6e54ebfc4a90b643a65e402c4048') + version("6.2.1", sha256="eae9326beb4158c386e39a356818031bd28f3124cf915f8c5b1dc4c7a36b4d7c") # FIXME: Add dependencies if required. - # depends_on('foo') + # depends_on("foo") def configure_args(self): # FIXME: Add arguments other than --prefix @@ -154,15 +273,16 @@ generates a boilerplate template for your package, and opens up the new return args The tedious stuff (creating the class, checksumming archives) has been -done for you. You'll notice that ``spack create`` correctly detected that -``gmp`` uses the Autotools build system. It created a new ``Gmp`` package -that subclasses the ``AutotoolsPackage`` base class. This base class -provides basic installation methods common to all Autotools packages: +done for you. Spack correctly detected that ``gmp`` uses the ``autotools`` +build system, so it created a new ``Gmp`` package that subclasses the +``AutotoolsPackage`` base class. + +The default installation procedure for a package subclassing the ``AutotoolsPackage`` +is to go through the typical process of: .. code-block:: bash ./configure --prefix=/path/to/installation/directory - make make check make install @@ -209,12 +329,14 @@ The rest of the tasks you need to do are as follows: Your new package may require specific flags during ``configure``. These can be added via ``configure_args``. Specifics will differ depending on the package and its build system. - :ref:`Implementing the install method <install-method>` is + :ref:`installation_process` is covered in detail later. -Passing a URL to ``spack create`` is a convenient and easy way to get -a basic package template, but what if your software is licensed and -cannot be downloaded from a URL? You can still create a boilerplate +^^^^^^^^^^^^^^^^^^^^^^^^^ +Non-downloadable software +^^^^^^^^^^^^^^^^^^^^^^^^^ + +If your software cannot be downloaded from a URL you can still create a boilerplate ``package.py`` by telling ``spack create`` what name you want to use: .. code-block:: console @@ -223,40 +345,23 @@ cannot be downloaded from a URL? You can still create a boilerplate This will create a simple ``intel`` package with an ``install()`` method that you can craft to install your package. - -What if ``spack create <url>`` guessed the wrong name or build system? -For example, if your package uses the Autotools build system but does -not come with a ``configure`` script, Spack won't realize it uses -Autotools. You can overwrite the old package with ``--force`` and specify -a name with ``--name`` or a build system template to use with ``--template``: +Likewise, you can force the build system to be used with ``--template`` and, +in case it's needed, you can overwrite a package already in the repository +with ``--force``: .. code-block:: console $ spack create --name gmp https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2 $ spack create --force --template autotools https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2 -.. note:: - - If you are creating a package that uses the Autotools build system - but does not come with a ``configure`` script, you'll need to add an - ``autoreconf`` method to your package that explains how to generate - the ``configure`` script. You may also need the following dependencies: - - .. code-block:: python - - depends_on('autoconf', type='build') - depends_on('automake', type='build') - depends_on('libtool', type='build') - depends_on('m4', type='build') - A complete list of available build system templates can be found by running ``spack create --help``. .. _cmd-spack-edit: -^^^^^^^^^^^^^^ -``spack edit`` -^^^^^^^^^^^^^^ +------------------------- +Editing existing packages +------------------------- One of the easiest ways to learn how to write packages is to look at existing ones. You can edit a package file by name with the ``spack @@ -266,10 +371,15 @@ edit`` command: $ spack edit gmp -So, if you used ``spack create`` to create a package, then saved and -closed the resulting file, you can get back to it with ``spack edit``. -The ``gmp`` package actually lives in -``$SPACK_ROOT/var/spack/repos/builtin/packages/gmp/package.py``, +If you used ``spack create`` to create a package, you can get back to +it later with ``spack edit``. For instance, the ``gmp`` package actually +lives in: + +.. code-block:: console + + $ spack location -p gmp + ${SPACK_ROOT}/var/spack/repos/builtin/packages/gmp/package.py + but ``spack edit`` provides a much simpler shortcut and saves you the trouble of typing the full path. @@ -2422,7 +2532,7 @@ Spack provides a mechanism for dependencies to influence the environment of their dependents by overriding the :meth:`setup_dependent_run_environment <spack.package_base.PackageBase.setup_dependent_run_environment>` or the -:meth:`setup_dependent_build_environment <spack.package_base.PackageBase.setup_dependent_build_environment>` +:meth:`setup_dependent_build_environment <spack.builder.Builder.setup_dependent_build_environment>` methods. The Qt package, for instance, uses this call: @@ -3280,67 +3390,91 @@ the Python extensions provided by them: once for ``+python`` and once for ``~python``. Other than using a little extra disk space, that solution has no serious problems. -.. _installation_procedure: +.. _installation_process: + +-------------------------------- +Overriding build system defaults +-------------------------------- ---------------------------------------- -Implementing the installation procedure ---------------------------------------- +.. note:: -The last element of a package is its **installation procedure**. This is -where the real work of installation happens, and it's the main part of -the package you'll need to customize for each piece of software. + If you code a single class in ``package.py`` all the functions shown in the table below + can be implemented with the same signature on the ``*Package`` instead of the corresponding builder. -Defining an installation procedure means overriding a set of methods or attributes -that will be called at some point during the installation of the package. -The package base class, usually specialized for a given build system, determines the -actual set of entities available for overriding. -The classes that are currently provided by Spack are: + +Most of the time the default implementation of methods or attributes in build system base classes +is what a packager needs, and just a very few entities need to be overwritten. Typically we just +need to override methods like ``configure_args``: + +.. code-block:: python + + def configure_args(self): + args = ["--enable-cxx"] + self.enable_or_disable("libs") + if "libs=static" in self.spec: + args.append("--with-pic") + return args + +The actual set of entities available for overriding in ``package.py`` depend on +the build system. The build systems currently supported by Spack are: +----------------------------------------------------------+----------------------------------+ -| **Base Class** | **Purpose** | +| **API docs** | **Description** | +==========================================================+==================================+ -| :class:`~spack.package_base.Package` | General base class not | -| | specialized for any build system | +| :class:`~spack.build_systems.generic` | Generic build system without any | +| | base implementation | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.makefile.MakefilePackage` | Specialized class for packages | -| | built invoking | +| :class:`~spack.build_systems.makefile` | Specialized build system for | +| | software built invoking | | | hand-written Makefiles | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.autotools.AutotoolsPackage` | Specialized class for packages | -| | built using GNU Autotools | +| :class:`~spack.build_systems.autotools` | Specialized build system for | +| | software built using | +| | GNU Autotools | ++----------------------------------------------------------+----------------------------------+ +| :class:`~spack.build_systems.cmake` | Specialized build system for | +| | software built using CMake | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.cmake.CMakePackage` | Specialized class for packages | -| | built using CMake | +| :class:`~spack.build_systems.maven` | Specialized build system for | +| | software built using Maven | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.cuda.CudaPackage` | A helper class for packages that | -| | use CUDA | +| :class:`~spack.build_systems.meson` | Specialized build system for | +| | software built using Meson | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.qmake.QMakePackage` | Specialized class for packages | -| | built using QMake | +| :class:`~spack.build_systems.nmake` | Specialized build system for | +| | software built using NMake | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.rocm.ROCmPackage` | A helper class for packages that | -| | use ROCm | +| :class:`~spack.build_systems.qmake` | Specialized build system for | +| | software built using QMake | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.scons.SConsPackage` | Specialized class for packages | -| | built using SCons | +| :class:`~spack.build_systems.scons` | Specialized build system for | +| | software built using SCons | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.waf.WafPackage` | Specialized class for packages | -| | built using Waf | +| :class:`~spack.build_systems.waf` | Specialized build system for | +| | software built using Waf | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.r.RPackage` | Specialized class for | +| :class:`~spack.build_systems.r` | Specialized build system for | | | R extensions | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.octave.OctavePackage` | Specialized class for | +| :class:`~spack.build_systems.octave` | Specialized build system for | | | Octave packages | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.python.PythonPackage` | Specialized class for | +| :class:`~spack.build_systems.python` | Specialized build system for | | | Python extensions | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.perl.PerlPackage` | Specialized class for | +| :class:`~spack.build_systems.perl` | Specialized build system for | | | Perl extensions | +----------------------------------------------------------+----------------------------------+ -| :class:`~spack.build_systems.intel.IntelPackage` | Specialized class for licensed | -| | Intel software | +| :class:`~spack.build_systems.ruby` | Specialized build system for | +| | Ruby extensions | ++----------------------------------------------------------+----------------------------------+ +| :class:`~spack.build_systems.intel` | Specialized build system for | +| | licensed Intel software | ++----------------------------------------------------------+----------------------------------+ +| :class:`~spack.build_systems.oneapi` | Specialized build system for | +| | Intel onaAPI software | ++----------------------------------------------------------+----------------------------------+ +| :class:`~spack.build_systems.aspell_dict` | Specialized build system for | +| | Aspell dictionaries | +----------------------------------------------------------+----------------------------------+ @@ -3353,69 +3487,17 @@ The classes that are currently provided by Spack are: For example, a Python extension installed with CMake would ``extends('python')`` and subclass from :class:`~spack.build_systems.cmake.CMakePackage`. -^^^^^^^^^^^^^^^^^^^^^ -Installation pipeline -^^^^^^^^^^^^^^^^^^^^^ - -When a user runs ``spack install``, Spack: - -1. Fetches an archive for the correct version of the software. -2. Expands the archive. -3. Sets the current working directory to the root directory of the expanded archive. - -Then, depending on the base class of the package under consideration, it will execute -a certain number of **phases** that reflect the way a package of that type is usually built. -The name and order in which the phases will be executed can be obtained either reading the API -docs at :py:mod:`~.spack.build_systems`, or using the ``spack info`` command: - -.. code-block:: console - :emphasize-lines: 26-27 - - $ spack info --phases m4 - AutotoolsPackage: m4 - - Description: - GNU M4 is an implementation of the traditional Unix macro processor. - - Homepage: https://www.gnu.org/software/m4/m4.html - - Preferred version: - 1.4.19 https://ftpmirror.gnu.org/m4/m4-1.4.19.tar.gz - - Safe versions: - 1.4.19 https://ftpmirror.gnu.org/m4/m4-1.4.19.tar.gz - 1.4.18 https://ftpmirror.gnu.org/m4/m4-1.4.18.tar.gz - 1.4.17 https://ftpmirror.gnu.org/m4/m4-1.4.17.tar.gz - - Deprecated versions: - None - - Variants: - Name [Default] When Allowed values Description - ============== ==== ============== =============================== - - sigsegv [on] -- on, off Build the libsigsegv dependency - - Installation Phases: - autoreconf configure build install - - Build Dependencies: - diffutils gnuconfig libsigsegv - - Link Dependencies: - libsigsegv - - Run Dependencies: - None - +^^^^^^^^^^^^^^^^^^^^^^^^^^ +Overriding builder methods +^^^^^^^^^^^^^^^^^^^^^^^^^^ -Typically, phases have default implementations that fit most of the common cases: +Build-system "phases" have default implementations that fit most of the common cases: .. literalinclude:: _spack_root/lib/spack/spack/build_systems/autotools.py - :pyobject: AutotoolsPackage.configure + :pyobject: AutotoolsBuilder.configure :linenos: -It is thus just sufficient for a packager to override a few +It is usually sufficient for a packager to override a few build system specific helper methods or attributes to provide, for instance, configure arguments: @@ -3423,31 +3505,31 @@ configure arguments: :pyobject: M4.configure_args :linenos: -.. note:: - Each specific build system has a list of attributes that can be overridden to - fine-tune the installation of a package without overriding an entire phase. To - have more information on them the place to go is the API docs of the :py:mod:`~.spack.build_systems` - module. +Each specific build system has a list of attributes and methods that can be overridden to +fine-tune the installation of a package without overriding an entire phase. To +have more information on them the place to go is the API docs of the :py:mod:`~.spack.build_systems` +module. ^^^^^^^^^^^^^^^^^^^^^^^^^^ Overriding an entire phase ^^^^^^^^^^^^^^^^^^^^^^^^^^ -In extreme cases it may be necessary to override an entire phase. Regardless -of the build system, the signature is the same. For example, the signature -for the install phase is: +Sometimes it is necessary to override an entire phase. If the ``package.py`` contains +a single class recipe, see :ref:`package_class_structure`, then the signature for a +phase is: .. code-block:: python - class Foo(Package): + class Openjpeg(CMakePackage): def install(self, spec, prefix): ... +regardless of the build system. The arguments for the phase are: + ``self`` - For those not used to Python instance methods, this is the - package itself. In this case it's an instance of ``Foo``, which - extends ``Package``. For API docs on Package objects, see - :py:class:`Package <spack.package_base.Package>`. + This is the package object, which extends ``CMakePackage``. + For API docs on Package objects, see + :py:class:`Package <spack.package_base.PackageBase>`. ``spec`` This is the concrete spec object created by Spack from an @@ -3462,12 +3544,111 @@ for the install phase is: The arguments ``spec`` and ``prefix`` are passed only for convenience, as they always correspond to ``self.spec`` and ``self.spec.prefix`` respectively. -As mentioned in :ref:`install-environment`, you will usually not need to refer -to dependencies explicitly in your package file, as the compiler wrappers take care of most of -the heavy lifting here. There will be times, though, when you need to refer to -the install locations of dependencies, or when you need to do something different -depending on the version, compiler, dependencies, etc. that your package is -built with. These parameters give you access to this type of information. +If the ``package.py`` encodes builders explicitly, the signature for a phase changes slightly: + +.. code-block:: python + + class CMakeBuilder(spack.build_systems.cmake.CMakeBuilder): + def install(self, pkg, spec, prefix): + ... + +In this case the package is passed as the second argument, and ``self`` is the builder instance. + +.. _multiple_build_systems: + +^^^^^^^^^^^^^^^^^^^^^^ +Multiple build systems +^^^^^^^^^^^^^^^^^^^^^^ + +There are cases where a software actively supports two build systems, or changes build systems +as it evolves, or needs different build systems on different platforms. Spack allows dealing with +these cases natively, if a recipe is written using builders explicitly. + +For instance, software that supports two build systems unconditionally should derive from +both ``*Package`` base classes, and declare the possible use of multiple build systems using +a directive: + +.. code-block:: python + + class ArpackNg(CMakePackage, AutotoolsPackage): + + build_system("cmake", "autotools", default="cmake") + +In this case the software can be built with both ``autotools`` and ``cmake``. Since the package +supports multiple build systems, it is necessary to declare which one is the default. The ``package.py`` +will likely contain some overriding of default builder methods: + +.. code-block:: python + + class CMakeBuilder(spack.build_systems.cmake.CMakeBuilder): + def cmake_args(self): + pass + + class Autotoolsbuilder(spack.build_systems.autotools.AutotoolsBuilder): + def configure_args(self): + pass + +In more complex cases it might happen that the build system changes according to certain conditions, +for instance across versions. That can be expressed with conditional variant values: + +.. code-block:: python + + class ArpackNg(CMakePackage, AutotoolsPackage): + + build_system( + conditional("cmake", when="@0.64:"), + conditional("autotools", when="@:0.63"), + default="cmake", + ) + +In the example the directive impose a change from ``Autotools`` to ``CMake`` going +from ``v0.63`` to ``v0.64``. + +^^^^^^^^^^^^^^^^^^ +Mixin base classes +^^^^^^^^^^^^^^^^^^ + +Besides build systems, there are other cases where common metadata and behavior can be extracted +and reused by many packages. For instance, packages that depend on ``Cuda`` or ``Rocm``, share +common dependencies and constraints. To factor these attributes into a single place, Spack provides +a few mixin classes in the ``spack.build_systems`` module: + ++---------------------------------------------------------------+----------------------------------+ +| **API docs** | **Description** | ++===============================================================+==================================+ +| :class:`~spack.build_systems.cuda.CudaPackage` | A helper class for packages that | +| | use CUDA | ++---------------------------------------------------------------+----------------------------------+ +| :class:`~spack.build_systems.rocm.ROCmPackage` | A helper class for packages that | +| | use ROCm | ++---------------------------------------------------------------+----------------------------------+ +| :class:`~spack.build_systems.gnu.GNUMirrorPackage` | A helper class for GNU packages | ++---------------------------------------------------------------+----------------------------------+ +| :class:`~spack.build_systems.python.PythonExtension` | A helper class for Python | +| | extensions | ++---------------------------------------------------------------+----------------------------------+ +| :class:`~spack.build_systems.sourceforge.SourceforgePackage` | A helper class for packages | +| | from sourceforge.org | ++---------------------------------------------------------------+----------------------------------+ +| :class:`~spack.build_systems.sourceware.SourcewarePackage` | A helper class for packages | +| | from sourceware.org | ++---------------------------------------------------------------+----------------------------------+ +| :class:`~spack.build_systems.xorg.XorgPackage` | A helper class for x.org | +| | packages | ++---------------------------------------------------------------+----------------------------------+ + +These classes should be used by adding them to the inheritance tree of the package that needs them, +for instance: + +.. code-block:: python + + class Cp2k(MakefilePackage, CudaPackage): + """CP2K is a quantum chemistry and solid state physics software package + that can perform atomistic simulations of solid state, liquid, molecular, + periodic, material, crystal, and biological systems + """ + +In the example above ``Cp2k`` inherits all the conflicts and variants that ``CudaPackage`` defines. .. _install-environment: @@ -6116,3 +6297,82 @@ might write: DWARF_PREFIX = $(spack location --install-dir libdwarf) CXXFLAGS += -I$DWARF_PREFIX/include CXXFLAGS += -L$DWARF_PREFIX/lib + + +.. _package_class_structure: + +-------------------------- +Package class architecture +-------------------------- + +.. note:: + + This section aims to provide a high-level knowledge of how the package class architecture evolved + in Spack, and provides some insights on the current design. + +Packages in Spack were originally designed to support only a single build system. The overall +class structure for a package looked like: + +.. image:: images/original_package_architecture.png + :scale: 60 % + :align: center + +In this architecture the base class ``AutotoolsPackage`` was responsible for both the metadata +related to the ``autotools`` build system (e.g. dependencies or variants common to all packages +using it), and for encoding the default installation procedure. + +In reality, a non-negligible number of packages are either changing their build system during the evolution of the +project, or using different build systems for different platforms. An architecture based on a single class +requires hacks or other workarounds to deal with these cases. + +To support a model more adherent to reality, Spack v0.19 changed its internal design by extracting +the attributes and methods related to building a software into a separate hierarchy: + +.. image:: images/builder_package_architecture.png + :scale: 60 % + :align: center + +In this new format each ``package.py`` contains one ``*Package`` class that gathers all the metadata, +and one or more ``*Builder`` classes that encode the installation procedure. A specific builder object +is created just before the software is built, so at a time where Spack knows which build system needs +to be used for the current installation, and receives a ``package`` object during initialization. + +^^^^^^^^^^^^^^^^^^^^^^^^ +``build_system`` variant +^^^^^^^^^^^^^^^^^^^^^^^^ + +To allow imposing conditions based on the build system, each package must a have ``build_system`` variant, +which is usually inherited from base classes. This variant allows for writing metadata that is conditional +on the build system: + +.. code-block:: python + + with when("build_system=cmake"): + depends_on("cmake", type="build") + +and also for selecting a specific build system from a spec literal, like in the following command: + +.. code-block:: console + + $ spack install arpack-ng build_system=autotools + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Compatibility with single-class format +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Internally, Spack always uses builders to perform operations related to the installation of a specific software. +The builders are created in the ``spack.builder.create`` function + +.. literalinclude:: _spack_root/lib/spack/spack/builder.py + :pyobject: create + +To achieve backward compatibility with the single-class format Spack creates in this function a special +"adapter builder", if no custom builder is detected in the recipe: + +.. image:: images/adapter.png + :scale: 60 % + :align: center + +Overall the role of the adapter is to route access to attributes of methods first through the ``*Package`` +hierarchy, and then back to the base class builder. This is schematically shown in the diagram above, where +the adapter role is to "emulate" a method resolution order like the one represented by the red arrows.
\ No newline at end of file diff --git a/lib/spack/spack/audit.py b/lib/spack/spack/audit.py index ee974a19b1..de9fc1a05b 100644 --- a/lib/spack/spack/audit.py +++ b/lib/spack/spack/audit.py @@ -503,6 +503,33 @@ def _ensure_all_packages_use_sha256_checksums(pkgs, error_cls): return errors +@package_properties +def _ensure_env_methods_are_ported_to_builders(pkgs, error_cls): + """Ensure that methods modifying the build environment are ported to builder classes.""" + errors = [] + for pkg_name in pkgs: + pkg_cls = spack.repo.path.get_pkg_class(pkg_name) + buildsystem_variant, _ = pkg_cls.variants["build_system"] + buildsystem_names = [getattr(x, "value", x) for x in buildsystem_variant.values] + builder_cls_names = [spack.builder.BUILDER_CLS[x].__name__ for x in buildsystem_names] + module = pkg_cls.module + has_builders_in_package_py = any( + getattr(module, name, False) for name in builder_cls_names + ) + if not has_builders_in_package_py: + continue + + for method_name in ("setup_build_environment", "setup_dependent_build_environment"): + if hasattr(pkg_cls, method_name): + msg = ( + "Package '{}' need to move the '{}' method from the package class to the" + " appropriate builder class".format(pkg_name, method_name) + ) + errors.append(error_cls(msg, [])) + + return errors + + @package_https_directives def _linting_package_file(pkgs, error_cls): """Check for correctness of links""" @@ -660,7 +687,13 @@ def _ensure_variant_defaults_are_parsable(pkgs, error_cls): errors.append(error_cls(error_msg.format(variant_name, pkg_name), [])) continue - vspec = variant.make_default() + try: + vspec = variant.make_default() + except spack.variant.MultipleValuesInExclusiveVariantError: + error_msg = "Cannot create a default value for the variant '{}' in package '{}'" + errors.append(error_cls(error_msg.format(variant_name, pkg_name), [])) + continue + try: variant.validate_or_raise(vspec, pkg_cls=pkg_cls) except spack.variant.InvalidVariantValueError: diff --git a/lib/spack/spack/build_environment.py b/lib/spack/spack/build_environment.py index f8fda997d3..9fe7c1fbb7 100644 --- a/lib/spack/spack/build_environment.py +++ b/lib/spack/spack/build_environment.py @@ -52,6 +52,7 @@ from llnl.util.tty.log import MultiProcessFd import spack.build_systems.cmake import spack.build_systems.meson +import spack.builder import spack.config import spack.install_test import spack.main @@ -558,9 +559,9 @@ def _set_variables_for_single_module(pkg, module): if sys.platform == "win32": m.nmake = Executable("nmake") # Standard CMake arguments - m.std_cmake_args = spack.build_systems.cmake.CMakePackage._std_args(pkg) - m.std_meson_args = spack.build_systems.meson.MesonPackage._std_args(pkg) - m.std_pip_args = spack.build_systems.python.PythonPackage._std_args(pkg) + m.std_cmake_args = spack.build_systems.cmake.CMakeBuilder.std_args(pkg) + m.std_meson_args = spack.build_systems.meson.MesonBuilder.std_args(pkg) + m.std_pip_args = spack.build_systems.python.PythonPipBuilder.std_args(pkg) # Put spack compiler paths in module scope. link_dir = spack.paths.build_env_path @@ -727,38 +728,6 @@ def get_rpaths(pkg): return list(dedupe(filter_system_paths(rpaths))) -def get_std_cmake_args(pkg): - """List of standard arguments used if a package is a CMakePackage. - - Returns: - list: standard arguments that would be used if this - package were a CMakePackage instance. - - Args: - pkg (spack.package_base.PackageBase): package under consideration - - Returns: - list: arguments for cmake - """ - return spack.build_systems.cmake.CMakePackage._std_args(pkg) - - -def get_std_meson_args(pkg): - """List of standard arguments used if a package is a MesonPackage. - - Returns: - list: standard arguments that would be used if this - package were a MesonPackage instance. - - Args: - pkg (spack.package_base.PackageBase): package under consideration - - Returns: - list: arguments for meson - """ - return spack.build_systems.meson.MesonPackage._std_args(pkg) - - def parent_class_modules(cls): """ Get list of superclass modules that descend from spack.package_base.PackageBase @@ -819,7 +788,8 @@ def setup_package(pkg, dirty, context="build"): platform.setup_platform_environment(pkg, env_mods) if context == "build": - pkg.setup_build_environment(env_mods) + builder = spack.builder.create(pkg) + builder.setup_build_environment(env_mods) if (not dirty) and (not env_mods.is_unset("CPATH")): tty.debug( @@ -1015,7 +985,8 @@ def modifications_from_dependencies( module.__dict__.update(changes.__dict__) if context == "build": - dpkg.setup_dependent_build_environment(env, spec) + builder = spack.builder.create(dpkg) + builder.setup_dependent_build_environment(env, spec) else: dpkg.setup_dependent_run_environment(env, spec) @@ -1117,8 +1088,20 @@ def _setup_pkg_and_run( pkg.test_suite.stage, spack.install_test.TestSuite.test_log_name(pkg.spec) ) + error_msg = str(exc) + if isinstance(exc, (spack.multimethod.NoSuchMethodError, AttributeError)): + error_msg = ( + "The '{}' package cannot find an attribute while trying to build " + "from sources. This might be due to a change in Spack's package format " + "to support multiple build-systems for a single package. You can fix this " + "by updating the build recipe, and you can also report the issue as a bug. " + "More information at https://spack.readthedocs.io/en/latest/packaging_guide.html#installation-procedure" + ).format(pkg.name) + error_msg = colorize("@*R{{{}}}".format(error_msg)) + error_msg = "{}\n\n{}".format(str(exc), error_msg) + # make a pickleable exception to send to parent. - msg = "%s: %s" % (exc_type.__name__, str(exc)) + msg = "%s: %s" % (exc_type.__name__, error_msg) ce = ChildError( msg, diff --git a/lib/spack/spack/build_systems/_checks.py b/lib/spack/spack/build_systems/_checks.py new file mode 100644 index 0000000000..73d5bbdb93 --- /dev/null +++ b/lib/spack/spack/build_systems/_checks.py @@ -0,0 +1,124 @@ +# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other +# Spack Project Developers. See the top-level COPYRIGHT file for details. +# +# SPDX-License-Identifier: (Apache-2.0 OR MIT) +import os + +import six + +import llnl.util.lang + +import spack.builder +import spack.installer +import spack.relocate +import spack.store + + +def sanity_check_prefix(builder): + """Check that specific directories and files are created after installation. + + The files to be checked are in the ``sanity_check_is_file`` attribute of the + package object, while the directories are in the ``sanity_check_is_dir``. + + Args: + builder (spack.builder.Builder): builder that installed the package + """ + pkg = builder.pkg + + def check_paths(path_list, filetype, predicate): + if isinstance(path_list, six.string_types): + path_list = [path_list] + + for path in path_list: + abs_path = os.path.join(pkg.prefix, path) + if not predicate(abs_path): + msg = "Install failed for {0}. No such {1} in prefix: {2}" + msg = msg.format(pkg.name, filetype, path) + raise spack.installer.InstallError(msg) + + check_paths(pkg.sanity_check_is_file, "file", os.path.isfile) + check_paths(pkg.sanity_check_is_dir, "directory", os.path.isdir) + + ignore_file = llnl.util.lang.match_predicate(spack.store.layout.hidden_file_regexes) + if all(map(ignore_file, os.listdir(pkg.prefix))): + msg = "Install failed for {0}. Nothing was installed!" + raise spack.installer.InstallError(msg.format(pkg.name)) + + +def apply_macos_rpath_fixups(builder): + """On Darwin, make installed libraries more easily relocatable. + + Some build systems (handrolled, autotools, makefiles) can set their own + rpaths that are duplicated by spack's compiler wrapper. This fixup + interrogates, and postprocesses if necessary, all libraries installed + by the code. + + It should be added as a @run_after to packaging systems (or individual + packages) that do not install relocatable libraries by default. + + Args: + builder (spack.builder.Builder): builder that installed the package + """ + spack.relocate.fixup_macos_rpaths(builder.spec) + + +def ensure_build_dependencies_or_raise(spec, dependencies, error_msg): + """Ensure that some build dependencies are present in the concrete spec. + + If not, raise a RuntimeError with a helpful error message. + + Args: + spec (spack.spec.Spec): concrete spec to be checked. + dependencies (list of spack.spec.Spec): list of abstract specs to be satisfied + error_msg (str): brief error message to be prepended to a longer description + + Raises: + RuntimeError: when the required build dependencies are not found + """ + assert spec.concrete, "Can ensure build dependencies only on concrete specs" + build_deps = [d.name for d in spec.dependencies(deptype="build")] + missing_deps = [x for x in dependencies if x not in build_deps] + + if not missing_deps: + return + + # Raise an exception on missing deps. + msg = ( + "{0}: missing dependencies: {1}.\n\nPlease add " + "the following lines to the package:\n\n".format(error_msg, ", ".join(missing_deps)) + ) + + for dep in missing_deps: + msg += " depends_on('{0}', type='build', when='@{1} {2}')\n".format( + dep, spec.version, "build_system=autotools" + ) + + msg += "\nUpdate the version (when='@{0}') as needed.".format(spec.version) + raise RuntimeError(msg) + + +def execute_build_time_tests(builder): + """Execute the build-time tests prescribed by builder. + + Args: + builder (Builder): builder prescribing the test callbacks. The name of the callbacks is + stored as a list of strings in the ``build_time_test_callbacks`` attribute. + """ + builder.pkg.run_test_callbacks(builder, builder.build_time_test_callbacks, "build") + + +def execute_install_time_tests(builder): + """Execute the install-time tests prescribed by builder. + + Args: + builder (Builder): builder prescribing the test callbacks. The name of the callbacks is + stored as a list of strings in the ``install_time_test_callbacks`` attribute. + """ + builder.pkg.run_test_callbacks(builder, builder.install_time_test_callbacks, "install") + + +class BaseBuilder(spack.builder.Builder): + """Base class for builders to register common checks""" + + # Check that self.prefix is there after installation + spack.builder.run_after("install")(sanity_check_prefix) diff --git a/lib/spack/spack/build_systems/aspell_dict.py b/lib/spack/spack/build_systems/aspell_dict.py index f1e41cc3df..9de8255e68 100644 --- a/lib/spack/spack/build_systems/aspell_dict.py +++ b/lib/spack/spack/build_systems/aspell_dict.py @@ -2,18 +2,36 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) +import llnl.util.filesystem as fs -# Why doesn't this work for me? -# from spack import * -from llnl.util.filesystem import filter_file +import spack.directives +import spack.package_base +import spack.util.executable -from spack.build_systems.autotools import AutotoolsPackage -from spack.directives import extends -from spack.package_base import ExtensionError -from spack.util.executable import which +from .autotools import AutotoolsBuilder, AutotoolsPackage + + +class AspellBuilder(AutotoolsBuilder): + """The Aspell builder is close enough to an autotools builder to allow + specializing the builder class, so to use variables that are specific + to the Aspell extensions. + """ + + def configure(self, pkg, spec, prefix): + aspell = spec["aspell"].prefix.bin.aspell + prezip = spec["aspell"].prefix.bin.prezip + destdir = prefix + + sh = spack.util.executable.which("sh") + sh( + "./configure", + "--vars", + "ASPELL={0}".format(aspell), + "PREZIP={0}".format(prezip), + "DESTDIR={0}".format(destdir), + ) -# # Aspell dictionaries install their bits into their prefix.lib # and when activated they'll get symlinked into the appropriate aspell's # dict dir (see aspell's {de,}activate methods). @@ -23,12 +41,17 @@ from spack.util.executable import which class AspellDictPackage(AutotoolsPackage): """Specialized class for building aspell dictionairies.""" - extends("aspell") + spack.directives.extends("aspell", when="build_system=autotools") + + #: Override the default autotools builder + AutotoolsBuilder = AspellBuilder def view_destination(self, view): aspell_spec = self.spec["aspell"] if view.get_projection_for_spec(aspell_spec) != aspell_spec.prefix: - raise ExtensionError("aspell does not support non-global extensions") + raise spack.package_base.ExtensionError( + "aspell does not support non-global extensions" + ) aspell = aspell_spec.command return aspell("dump", "config", "dict-dir", output=str).strip() @@ -36,19 +59,5 @@ class AspellDictPackage(AutotoolsPackage): return self.prefix.lib def patch(self): - filter_file(r"^dictdir=.*$", "dictdir=/lib", "configure") - filter_file(r"^datadir=.*$", "datadir=/lib", "configure") - - def configure(self, spec, prefix): - aspell = spec["aspell"].prefix.bin.aspell - prezip = spec["aspell"].prefix.bin.prezip - destdir = prefix - - sh = which("sh") - sh( - "./configure", - "--vars", - "ASPELL={0}".format(aspell), - "PREZIP={0}".format(prezip), - "DESTDIR={0}".format(destdir), - ) + fs.filter_file(r"^dictdir=.*$", "dictdir=/lib", "configure") + fs.filter_file(r"^datadir=.*$", "datadir=/lib", "configure") diff --git a/lib/spack/spack/build_systems/autotools.py b/lib/spack/spack/build_systems/autotools.py index 81a572de52..76e6bb27ce 100644 --- a/lib/spack/spack/build_systems/autotools.py +++ b/lib/spack/spack/build_systems/autotools.py @@ -6,87 +6,140 @@ import inspect import os import os.path import stat -from subprocess import PIPE, check_call +import subprocess from typing import List # novm import llnl.util.filesystem as fs import llnl.util.tty as tty -from llnl.util.filesystem import force_remove, working_dir -from spack.build_environment import InstallError -from spack.directives import conflicts, depends_on +import spack.build_environment +import spack.builder +import spack.package_base +from spack.directives import build_system, conflicts, depends_on +from spack.multimethod import when from spack.operating_systems.mac_os import macos_version -from spack.package_base import PackageBase, run_after, run_before from spack.util.executable import Executable from spack.version import Version +from ._checks import ( + BaseBuilder, + apply_macos_rpath_fixups, + ensure_build_dependencies_or_raise, + execute_build_time_tests, + execute_install_time_tests, +) -class AutotoolsPackage(PackageBase): - """Specialized class for packages built using GNU Autotools. - This class provides four phases that can be overridden: +class AutotoolsPackage(spack.package_base.PackageBase): + """Specialized class for packages built using GNU Autotools.""" - 1. :py:meth:`~.AutotoolsPackage.autoreconf` - 2. :py:meth:`~.AutotoolsPackage.configure` - 3. :py:meth:`~.AutotoolsPackage.build` - 4. :py:meth:`~.AutotoolsPackage.install` + #: This attribute is used in UI queries that need to know the build + #: system base class + build_system_class = "AutotoolsPackage" + + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "autotools" + + build_system("autotools") + + with when("build_system=autotools"): + depends_on("gnuconfig", type="build", when="target=ppc64le:") + depends_on("gnuconfig", type="build", when="target=aarch64:") + depends_on("gnuconfig", type="build", when="target=riscv64:") + conflicts("platform=windows") + + def flags_to_build_system_args(self, flags): + """Produces a list of all command line arguments to pass specified + compiler flags to configure.""" + # Has to be dynamic attribute due to caching. + setattr(self, "configure_flag_args", []) + for flag, values in flags.items(): + if values: + values_str = "{0}={1}".format(flag.upper(), " ".join(values)) + self.configure_flag_args.append(values_str) + # Spack's fflags are meant for both F77 and FC, therefore we + # additionaly set FCFLAGS if required. + values = flags.get("fflags", None) + if values: + values_str = "FCFLAGS={0}".format(" ".join(values)) + self.configure_flag_args.append(values_str) + + # Legacy methods (used by too many packages to change them, + # need to forward to the builder) + def enable_or_disable(self, *args, **kwargs): + return self.builder.enable_or_disable(*args, **kwargs) + + def with_or_without(self, *args, **kwargs): + return self.builder.with_or_without(*args, **kwargs) + + +@spack.builder.builder("autotools") +class AutotoolsBuilder(BaseBuilder): + """The autotools builder encodes the default way of installing software built + with autotools. It has four phases that can be overridden, if need be: + + 1. :py:meth:`~.AutotoolsBuilder.autoreconf` + 2. :py:meth:`~.AutotoolsBuilder.configure` + 3. :py:meth:`~.AutotoolsBuilder.build` + 4. :py:meth:`~.AutotoolsBuilder.install` + + They all have sensible defaults and for many packages the only thing necessary + is to override the helper method + :meth:`~spack.build_systems.autotools.AutotoolsBuilder.configure_args`. - They all have sensible defaults and for many packages the only thing - necessary will be to override the helper method - :meth:`~spack.build_systems.autotools.AutotoolsPackage.configure_args`. For a finer tuning you may also override: +-----------------------------------------------+--------------------+ | **Method** | **Purpose** | +===============================================+====================+ - | :py:attr:`~.AutotoolsPackage.build_targets` | Specify ``make`` | + | :py:attr:`~.AutotoolsBuilder.build_targets` | Specify ``make`` | | | targets for the | | | build phase | +-----------------------------------------------+--------------------+ - | :py:attr:`~.AutotoolsPackage.install_targets` | Specify ``make`` | + | :py:attr:`~.AutotoolsBuilder.install_targets` | Specify ``make`` | | | targets for the | | | install phase | +-----------------------------------------------+--------------------+ - | :py:meth:`~.AutotoolsPackage.check` | Run build time | + | :py:meth:`~.AutotoolsBuilder.check` | Run build time | | | tests if required | +-----------------------------------------------+--------------------+ """ #: Phases of a GNU Autotools package - phases = ["autoreconf", "configure", "build", "install"] - #: This attribute is used in UI queries that need to know the build - #: system base class - build_system_class = "AutotoolsPackage" - - @property - def patch_config_files(self): - """ - Whether or not to update old ``config.guess`` and ``config.sub`` files - distributed with the tarball. This currently only applies to - ``ppc64le:``, ``aarch64:``, and ``riscv64`` target architectures. The - substitutes are taken from the ``gnuconfig`` package, which is - automatically added as a build dependency for these architectures. In - case system versions of these config files are required, the - ``gnuconfig`` package can be marked external with a prefix pointing to - the directory containing the system ``config.guess`` and ``config.sub`` - files. - """ - return ( - self.spec.satisfies("target=ppc64le:") - or self.spec.satisfies("target=aarch64:") - or self.spec.satisfies("target=riscv64:") - ) - - #: Whether or not to update ``libtool`` - #: (currently only for Arm/Clang/Fujitsu/NVHPC compilers) + phases = ("autoreconf", "configure", "build", "install") + + #: Names associated with package methods in the old build-system format + legacy_methods = ( + "configure_args", + "check", + "installcheck", + ) + + #: Names associated with package attributes in the old build-system format + legacy_attributes = ( + "archive_files", + "patch_libtool", + "build_targets", + "install_targets", + "build_time_test_callbacks", + "install_time_test_callbacks", + "force_autoreconf", + "autoreconf_extra_args", + "install_libtool_archives", + "patch_config_files", + "configure_directory", + "configure_abs_path", + "build_directory", + "autoreconf_search_path_args", + ) + + #: Whether to update ``libtool`` (e.g. for Arm/Clang/Fujitsu/NVHPC compilers) patch_libtool = True - #: Targets for ``make`` during the :py:meth:`~.AutotoolsPackage.build` - #: phase + #: Targets for ``make`` during the :py:meth:`~.AutotoolsBuilder.build` phase build_targets = [] # type: List[str] - #: Targets for ``make`` during the :py:meth:`~.AutotoolsPackage.install` - #: phase + #: Targets for ``make`` during the :py:meth:`~.AutotoolsBuilder.install` phase install_targets = ["install"] #: Callback names for build-time test @@ -97,24 +150,40 @@ class AutotoolsPackage(PackageBase): #: Set to true to force the autoreconf step even if configure is present force_autoreconf = False + #: Options to be passed to autoreconf when using the default implementation autoreconf_extra_args = [] # type: List[str] - #: If False deletes all the .la files in the prefix folder - #: after the installation. If True instead it installs them. + #: If False deletes all the .la files in the prefix folder after the installation. + #: If True instead it installs them. install_libtool_archives = False - depends_on("gnuconfig", type="build", when="target=ppc64le:") - depends_on("gnuconfig", type="build", when="target=aarch64:") - depends_on("gnuconfig", type="build", when="target=riscv64:") - conflicts("platform=windows") + @property + def patch_config_files(self): + """Whether to update old ``config.guess`` and ``config.sub`` files + distributed with the tarball. + + This currently only applies to ``ppc64le:``, ``aarch64:``, and + ``riscv64`` target architectures. + + The substitutes are taken from the ``gnuconfig`` package, which is + automatically added as a build dependency for these architectures. In case + system versions of these config files are required, the ``gnuconfig`` package + can be marked external, with a prefix pointing to the directory containing the + system ``config.guess`` and ``config.sub`` files. + """ + return ( + self.pkg.spec.satisfies("target=ppc64le:") + or self.pkg.spec.satisfies("target=aarch64:") + or self.pkg.spec.satisfies("target=riscv64:") + ) @property def _removed_la_files_log(self): - """File containing the list of remove libtool archives""" + """File containing the list of removed libtool archives""" build_dir = self.build_directory if not os.path.isabs(self.build_directory): - build_dir = os.path.join(self.stage.path, build_dir) + build_dir = os.path.join(self.pkg.stage.path, build_dir) return os.path.join(build_dir, "removed_la_files.txt") @property @@ -125,13 +194,13 @@ class AutotoolsPackage(PackageBase): files.append(self._removed_la_files_log) return files - @run_after("autoreconf") + @spack.builder.run_after("autoreconf") def _do_patch_config_files(self): - """Some packages ship with older config.guess/config.sub files and - need to have these updated when installed on a newer architecture. - In particular, config.guess fails for PPC64LE for version prior - to a 2013-06-10 build date (automake 1.13.4) and for ARM (aarch64) and - RISC-V (riscv64). + """Some packages ship with older config.guess/config.sub files and need to + have these updated when installed on a newer architecture. + + In particular, config.guess fails for PPC64LE for version prior to a + 2013-06-10 build date (automake 1.13.4) and for AArch64 and RISC-V. """ if not self.patch_config_files: return @@ -139,11 +208,11 @@ class AutotoolsPackage(PackageBase): # TODO: Expand this to select the 'config.sub'-compatible architecture # for each platform (e.g. 'config.sub' doesn't accept 'power9le', but # does accept 'ppc64le'). - if self.spec.satisfies("target=ppc64le:"): + if self.pkg.spec.satisfies("target=ppc64le:"): config_arch = "ppc64le" - elif self.spec.satisfies("target=aarch64:"): + elif self.pkg.spec.satisfies("target=aarch64:"): config_arch = "aarch64" - elif self.spec.satisfies("target=riscv64:"): + elif self.pkg.spec.satisfies("target=riscv64:"): config_arch = "riscv64" else: config_arch = "local" @@ -155,7 +224,7 @@ class AutotoolsPackage(PackageBase): args = [script_abs_path] + additional_args.get(script_name, []) try: - check_call(args, stdout=PIPE, stderr=PIPE) + subprocess.check_call(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE) except Exception as e: tty.debug(e) return False @@ -163,7 +232,7 @@ class AutotoolsPackage(PackageBase): return True # Get the list of files that needs to be patched - to_be_patched = fs.find(self.stage.path, files=["config.sub", "config.guess"]) + to_be_patched = fs.find(self.pkg.stage.path, files=["config.sub", "config.guess"]) to_be_patched = [f for f in to_be_patched if not runs_ok(f)] # If there are no files to be patched, return early @@ -171,22 +240,21 @@ class AutotoolsPackage(PackageBase): return # Otherwise, require `gnuconfig` to be a build dependency - self._require_build_deps( - pkgs=["gnuconfig"], spec=self.spec, err="Cannot patch config files" + ensure_build_dependencies_or_raise( + spec=self.pkg.spec, dependencies=["gnuconfig"], error_msg="Cannot patch config files" ) # Get the config files we need to patch (config.sub / config.guess). to_be_found = list(set(os.path.basename(f) for f in to_be_patched)) - gnuconfig = self.spec["gnuconfig"] + gnuconfig = self.pkg.spec["gnuconfig"] gnuconfig_dir = gnuconfig.prefix # An external gnuconfig may not not have a prefix. if gnuconfig_dir is None: - raise InstallError( - "Spack could not find substitutes for GNU config " - "files because no prefix is available for the " - "`gnuconfig` package. Make sure you set a prefix " - "path instead of modules for external `gnuconfig`." + raise spack.build_environment.InstallError( + "Spack could not find substitutes for GNU config files because no " + "prefix is available for the `gnuconfig` package. Make sure you set a " + "prefix path instead of modules for external `gnuconfig`." ) candidates = fs.find(gnuconfig_dir, files=to_be_found, recursive=False) @@ -203,7 +271,7 @@ class AutotoolsPackage(PackageBase): msg += ( " or the `gnuconfig` package prefix is misconfigured as" " an external package" ) - raise InstallError(msg) + raise spack.build_environment.InstallError(msg) # Filter working substitutes candidates = [f for f in candidates if runs_ok(f)] @@ -228,7 +296,9 @@ To resolve this problem, please try the following: and set the prefix to the directory containing the `config.guess` and `config.sub` files. """ - raise InstallError(msg.format(", ".join(to_be_found), self.name)) + raise spack.build_environment.InstallError( + msg.format(", ".join(to_be_found), self.name) + ) # Copy the good files over the bad ones for abs_path in to_be_patched: @@ -238,7 +308,7 @@ To resolve this problem, please try the following: fs.copy(substitutes[name], abs_path) os.chmod(abs_path, mode) - @run_before("configure") + @spack.builder.run_before("configure") def _patch_usr_bin_file(self): """On NixOS file is not available in /usr/bin/file. Patch configure scripts to use file from path.""" @@ -250,7 +320,7 @@ To resolve this problem, please try the following: with fs.keep_modification_time(*x.filenames): x.filter(regex="/usr/bin/file", repl="file", string=True) - @run_before("configure") + @spack.builder.run_before("configure") def _set_autotools_environment_variables(self): """Many autotools builds use a version of mknod.m4 that fails when running as root unless FORCE_UNSAFE_CONFIGURE is set to 1. @@ -261,11 +331,10 @@ To resolve this problem, please try the following: Without it, configure just fails halfway through, but it can still run things *before* this check. Forcing this just removes a nuisance -- this is not circumventing any real protection. - """ os.environ["FORCE_UNSAFE_CONFIGURE"] = "1" - @run_before("configure") + @spack.builder.run_before("configure") def _do_patch_libtool_configure(self): """Patch bugs that propagate from libtool macros into "configure" and further into "libtool". Note that patches that can be fixed by patching @@ -293,7 +362,7 @@ To resolve this problem, please try the following: # Support Libtool 2.4.2 and older: x.filter(regex=r'^(\s*test \$p = "-R")(; then\s*)$', repl=r'\1 || test x-l = x"$p"\2') - @run_after("configure") + @spack.builder.run_after("configure") def _do_patch_libtool(self): """If configure generates a "libtool" script that does not correctly detect the compiler (and patch_libtool is set), patch in the correct @@ -328,31 +397,33 @@ To resolve this problem, please try the following: markers[tag] = "LIBTOOL TAG CONFIG: {0}".format(tag.upper()) # Replace empty linker flag prefixes: - if self.compiler.name == "nag": + if self.pkg.compiler.name == "nag": # Nag is mixed with gcc and g++, which are recognized correctly. # Therefore, we change only Fortran values: for tag in ["fc", "f77"]: marker = markers[tag] x.filter( regex='^wl=""$', - repl='wl="{0}"'.format(self.compiler.linker_arg), + repl='wl="{0}"'.format(self.pkg.compiler.linker_arg), start_at="# ### BEGIN {0}".format(marker), stop_at="# ### END {0}".format(marker), ) else: - x.filter(regex='^wl=""$', repl='wl="{0}"'.format(self.compiler.linker_arg)) + x.filter(regex='^wl=""$', repl='wl="{0}"'.format(self.pkg.compiler.linker_arg)) # Replace empty PIC flag values: for cc, marker in markers.items(): x.filter( regex='^pic_flag=""$', - repl='pic_flag="{0}"'.format(getattr(self.compiler, "{0}_pic_flag".format(cc))), + repl='pic_flag="{0}"'.format( + getattr(self.pkg.compiler, "{0}_pic_flag".format(cc)) + ), start_at="# ### BEGIN {0}".format(marker), stop_at="# ### END {0}".format(marker), ) # Other compiler-specific patches: - if self.compiler.name == "fj": + if self.pkg.compiler.name == "fj": x.filter(regex="-nostdlib", repl="", string=True) rehead = r"/\S*/" for o in [ @@ -365,12 +436,12 @@ To resolve this problem, please try the following: "crtendS.o", ]: x.filter(regex=(rehead + o), repl="", string=True) - elif self.compiler.name == "dpcpp": + elif self.pkg.compiler.name == "dpcpp": # Hack to filter out spurious predep_objects when building with Intel dpcpp # (see https://github.com/spack/spack/issues/32863): x.filter(regex=r"^(predep_objects=.*)/tmp/conftest-[0-9A-Fa-f]+\.o", repl=r"\1") x.filter(regex=r"^(predep_objects=.*)/tmp/a-[0-9A-Fa-f]+\.o", repl=r"\1") - elif self.compiler.name == "nag": + elif self.pkg.compiler.name == "nag": for tag in ["fc", "f77"]: marker = markers[tag] start_at = "# ### BEGIN {0}".format(marker) @@ -446,11 +517,8 @@ To resolve this problem, please try the following: @property def configure_directory(self): - """Returns the directory where 'configure' resides. - - :return: directory where to find configure - """ - return self.stage.source_path + """Return the directory where 'configure' resides.""" + return self.pkg.stage.source_path @property def configure_abs_path(self): @@ -463,34 +531,12 @@ To resolve this problem, please try the following: """Override to provide another place to build the package""" return self.configure_directory - @run_before("autoreconf") + @spack.builder.run_before("autoreconf") def delete_configure_to_force_update(self): if self.force_autoreconf: - force_remove(self.configure_abs_path) - - def _require_build_deps(self, pkgs, spec, err): - """Require `pkgs` to be direct build dependencies of `spec`. Raises a - RuntimeError with a helpful error messages when any dep is missing.""" - - build_deps = [d.name for d in spec.dependencies(deptype="build")] - missing_deps = [x for x in pkgs if x not in build_deps] - - if not missing_deps: - return - - # Raise an exception on missing deps. - msg = ( - "{0}: missing dependencies: {1}.\n\nPlease add " - "the following lines to the package:\n\n".format(err, ", ".join(missing_deps)) - ) - - for dep in missing_deps: - msg += " depends_on('{0}', type='build', when='@{1}')\n".format(dep, spec.version) - - msg += "\nUpdate the version (when='@{0}') as needed.".format(spec.version) - raise RuntimeError(msg) + fs.force_remove(self.configure_abs_path) - def autoreconf(self, spec, prefix): + def autoreconf(self, pkg, spec, prefix): """Not needed usually, configure should be already there""" # If configure exists nothing needs to be done @@ -498,8 +544,10 @@ To resolve this problem, please try the following: return # Else try to regenerate it, which reuquires a few build dependencies - self._require_build_deps( - pkgs=["autoconf", "automake", "libtool"], spec=spec, err="Cannot generate configure" + ensure_build_dependencies_or_raise( + spec=spec, + dependencies=["autoconf", "automake", "libtool"], + error_msg="Cannot generate configure", ) tty.msg("Configure script not found: trying to generate it") @@ -507,8 +555,8 @@ To resolve this problem, please try the following: tty.warn("* If the default procedure fails, consider implementing *") tty.warn("* a custom AUTORECONF phase in the package *") tty.warn("*********************************************************") - with working_dir(self.configure_directory): - m = inspect.getmodule(self) + with fs.working_dir(self.configure_directory): + m = inspect.getmodule(self.pkg) # This line is what is needed most of the time # --install, --verbose, --force autoreconf_args = ["-ivf"] @@ -524,98 +572,66 @@ To resolve this problem, please try the following: spack dependencies.""" return _autoreconf_search_path_args(self.spec) - @run_after("autoreconf") + @spack.builder.run_after("autoreconf") def set_configure_or_die(self): - """Checks the presence of a ``configure`` file after the - autoreconf phase. If it is found sets a module attribute - appropriately, otherwise raises an error. + """Ensure the presence of a "configure" script, or raise. If the "configure" + is found, a module level attribute is set. - :raises RuntimeError: if a configure script is not found in - :py:meth:`~AutotoolsPackage.configure_directory` + Raises: + RuntimeError: if the "configure" script is not found """ - # Check if a configure script is there. If not raise a RuntimeError. + # Check if the "configure" script is there. If not raise a RuntimeError. if not os.path.exists(self.configure_abs_path): msg = "configure script not found in {0}" raise RuntimeError(msg.format(self.configure_directory)) # Monkey-patch the configure script in the corresponding module - inspect.getmodule(self).configure = Executable(self.configure_abs_path) + inspect.getmodule(self.pkg).configure = Executable(self.configure_abs_path) def configure_args(self): - """Produces a list containing all the arguments that must be passed to - configure, except ``--prefix`` which will be pre-pended to the list. - - :return: list of arguments for configure + """Return the list of all the arguments that must be passed to configure, + except ``--prefix`` which will be pre-pended to the list. """ return [] - def flags_to_build_system_args(self, flags): - """Produces a list of all command line arguments to pass specified - compiler flags to configure.""" - # Has to be dynamic attribute due to caching. - setattr(self, "configure_flag_args", []) - for flag, values in flags.items(): - if values: - values_str = "{0}={1}".format(flag.upper(), " ".join(values)) - self.configure_flag_args.append(values_str) - # Spack's fflags are meant for both F77 and FC, therefore we - # additionaly set FCFLAGS if required. - values = flags.get("fflags", None) - if values: - values_str = "FCFLAGS={0}".format(" ".join(values)) - self.configure_flag_args.append(values_str) - - def configure(self, spec, prefix): - """Runs configure with the arguments specified in - :meth:`~spack.build_systems.autotools.AutotoolsPackage.configure_args` - and an appropriately set prefix. + def configure(self, pkg, spec, prefix): + """Run "configure", with the arguments specified by the builder and an + appropriately set prefix. """ - options = getattr(self, "configure_flag_args", []) + options = getattr(self.pkg, "configure_flag_args", []) options += ["--prefix={0}".format(prefix)] options += self.configure_args() - with working_dir(self.build_directory, create=True): - inspect.getmodule(self).configure(*options) - - def setup_build_environment(self, env): - if self.spec.platform == "darwin" and macos_version() >= Version("11"): - # Many configure files rely on matching '10.*' for macOS version - # detection and fail to add flags if it shows as version 11. - env.set("MACOSX_DEPLOYMENT_TARGET", "10.16") + with fs.working_dir(self.build_directory, create=True): + inspect.getmodule(self.pkg).configure(*options) - def build(self, spec, prefix): - """Makes the build targets specified by - :py:attr:``~.AutotoolsPackage.build_targets`` - """ + def build(self, pkg, spec, prefix): + """Run "make" on the build targets specified by the builder.""" # See https://autotools.io/automake/silent.html params = ["V=1"] params += self.build_targets - with working_dir(self.build_directory): - inspect.getmodule(self).make(*params) + with fs.working_dir(self.build_directory): + inspect.getmodule(self.pkg).make(*params) - def install(self, spec, prefix): - """Makes the install targets specified by - :py:attr:``~.AutotoolsPackage.install_targets`` - """ - with working_dir(self.build_directory): - inspect.getmodule(self).make(*self.install_targets) + def install(self, pkg, spec, prefix): + """Run "make" on the install targets specified by the builder.""" + with fs.working_dir(self.build_directory): + inspect.getmodule(self.pkg).make(*self.install_targets) - run_after("build")(PackageBase._run_default_build_time_test_callbacks) + spack.builder.run_after("build")(execute_build_time_tests) def check(self): - """Searches the Makefile for targets ``test`` and ``check`` - and runs them if found. - """ - with working_dir(self.build_directory): - self._if_make_target_execute("test") - self._if_make_target_execute("check") + """Run "make" on the ``test`` and ``check`` targets, if found.""" + with fs.working_dir(self.build_directory): + self.pkg._if_make_target_execute("test") + self.pkg._if_make_target_execute("check") def _activate_or_not( self, name, activation_word, deactivation_word, activation_value=None, variant=None ): - """This function contains the current implementation details of - :meth:`~spack.build_systems.autotools.AutotoolsPackage.with_or_without` and - :meth:`~spack.build_systems.autotools.AutotoolsPackage.enable_or_disable`. + """This function contain the current implementation details of + :meth:`~spack.build_systems.autotools.AutotoolsBuilder.with_or_without` and + :meth:`~spack.build_systems.autotools.AutotoolsBuilder.enable_or_disable`. Args: name (str): name of the option that is being activated or not @@ -671,7 +687,7 @@ To resolve this problem, please try the following: Raises: KeyError: if name is not among known variants """ - spec = self.spec + spec = self.pkg.spec args = [] if activation_value == "prefix": @@ -681,16 +697,16 @@ To resolve this problem, please try the following: # Defensively look that the name passed as argument is among # variants - if variant not in self.variants: + if variant not in self.pkg.variants: msg = '"{0}" is not a variant of "{1}"' - raise KeyError(msg.format(variant, self.name)) + raise KeyError(msg.format(variant, self.pkg.name)) if variant not in spec.variants: return [] # Create a list of pairs. Each pair includes a configuration # option and whether or not that option is activated - variant_desc, _ = self.variants[variant] + variant_desc, _ = self.pkg.variants[variant] if set(variant_desc.values) == set((True, False)): # BoolValuedVariant carry information about a single option. # Nonetheless, for uniformity of treatment we'll package them @@ -718,14 +734,18 @@ To resolve this problem, please try the following: override_name = "{0}_or_{1}_{2}".format( activation_word, deactivation_word, option_value ) - line_generator = getattr(self, override_name, None) + line_generator = getattr(self, override_name, None) or getattr( + self.pkg, override_name, None + ) # If not available use a sensible default if line_generator is None: def _default_generator(is_activated): if is_activated: line = "--{0}-{1}".format(activation_word, option_value) - if activation_value is not None and activation_value(option_value): + if activation_value is not None and activation_value( + option_value + ): # NOQA=ignore=E501 line += "={0}".format(activation_value(option_value)) return line return "--{0}-{1}".format(deactivation_word, option_value) @@ -764,7 +784,7 @@ To resolve this problem, please try the following: def enable_or_disable(self, name, activation_value=None, variant=None): """Same as - :meth:`~spack.build_systems.autotools.AutotoolsPackage.with_or_without` + :meth:`~spack.build_systems.autotools.AutotoolsBuilder.with_or_without` but substitute ``with`` with ``enable`` and ``without`` with ``disable``. Args: @@ -781,19 +801,14 @@ To resolve this problem, please try the following: """ return self._activate_or_not(name, "enable", "disable", activation_value, variant) - run_after("install")(PackageBase._run_default_install_time_test_callbacks) + spack.builder.run_after("install")(execute_install_time_tests) def installcheck(self): - """Searches the Makefile for an ``installcheck`` target - and runs it if found. - """ - with working_dir(self.build_directory): - self._if_make_target_execute("installcheck") - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) + """Run "make" on the ``installcheck`` target, if found.""" + with fs.working_dir(self.build_directory): + self.pkg._if_make_target_execute("installcheck") - @run_after("install") + @spack.builder.run_after("install") def remove_libtool_archives(self): """Remove all .la files in prefix sub-folders if the package sets ``install_libtool_archives`` to be False. @@ -803,14 +818,20 @@ To resolve this problem, please try the following: return # Remove the files and create a log of what was removed - libtool_files = fs.find(str(self.prefix), "*.la", recursive=True) + libtool_files = fs.find(str(self.pkg.prefix), "*.la", recursive=True) with fs.safe_remove(*libtool_files): fs.mkdirp(os.path.dirname(self._removed_la_files_log)) with open(self._removed_la_files_log, mode="w") as f: f.write("\n".join(libtool_files)) + def setup_build_environment(self, env): + if self.spec.platform == "darwin" and macos_version() >= Version("11"): + # Many configure files rely on matching '10.*' for macOS version + # detection and fail to add flags if it shows as version 11. + env.set("MACOSX_DEPLOYMENT_TARGET", "10.16") + # On macOS, force rpaths for shared library IDs and remove duplicate rpaths - run_after("install")(PackageBase.apply_macos_rpath_fixups) + spack.builder.run_after("install", when="platform=darwin")(apply_macos_rpath_fixups) def _autoreconf_search_path_args(spec): diff --git a/lib/spack/spack/build_systems/bundle.py b/lib/spack/spack/build_systems/bundle.py new file mode 100644 index 0000000000..fad0ba4e14 --- /dev/null +++ b/lib/spack/spack/build_systems/bundle.py @@ -0,0 +1,31 @@ +# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other +# Spack Project Developers. See the top-level COPYRIGHT file for details. +# +# SPDX-License-Identifier: (Apache-2.0 OR MIT) +import spack.builder +import spack.directives +import spack.package_base + + +class BundlePackage(spack.package_base.PackageBase): + """General purpose bundle, or no-code, package class.""" + + #: This attribute is used in UI queries that require to know which + #: build-system class we are using + build_system_class = "BundlePackage" + + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "bundle" + + #: Bundle packages do not have associated source or binary code. + has_code = False + + spack.directives.build_system("bundle") + + +@spack.builder.builder("bundle") +class BundleBuilder(spack.builder.Builder): + phases = ("install",) + + def install(self, pkg, spec, prefix): + pass diff --git a/lib/spack/spack/build_systems/cached_cmake.py b/lib/spack/spack/build_systems/cached_cmake.py index 9ffd2a82ab..7caf2a1539 100644 --- a/lib/spack/spack/build_systems/cached_cmake.py +++ b/lib/spack/spack/build_systems/cached_cmake.py @@ -3,12 +3,14 @@ # # SPDX-License-Identifier: (Apache-2.0 OR MIT) import os +from typing import Tuple +import llnl.util.filesystem as fs import llnl.util.tty as tty -from llnl.util.filesystem import install, mkdirp -from spack.build_systems.cmake import CMakePackage -from spack.package_base import run_after +import spack.builder + +from .cmake import CMakeBuilder, CMakePackage def cmake_cache_path(name, value, comment=""): @@ -28,44 +30,46 @@ def cmake_cache_option(name, boolean_value, comment=""): return 'set({0} {1} CACHE BOOL "{2}")\n'.format(name, value, comment) -class CachedCMakePackage(CMakePackage): - """Specialized class for packages built using CMake initial cache. +class CachedCMakeBuilder(CMakeBuilder): - This feature of CMake allows packages to increase reproducibility, - especially between Spack- and manual builds. It also allows packages to - sidestep certain parsing bugs in extremely long ``cmake`` commands, and to - avoid system limits on the length of the command line.""" + #: Names associated with package methods in the old build-system format + legacy_methods = CMakeBuilder.legacy_methods + ( + "initconfig_compiler_entries", + "initconfig_mpi_entries", + "initconfig_hardware_entries", + "std_initconfig_entries", + "initconfig_package_entries", + ) # type: Tuple[str, ...] - phases = ["initconfig", "cmake", "build", "install"] + #: Names associated with package attributes in the old build-system format + legacy_attributes = CMakeBuilder.legacy_attributes + ( + "cache_name", + "cache_path", + ) # type: Tuple[str, ...] @property def cache_name(self): return "{0}-{1}-{2}@{3}.cmake".format( - self.name, - self.spec.architecture, - self.spec.compiler.name, - self.spec.compiler.version, + self.pkg.name, + self.pkg.spec.architecture, + self.pkg.spec.compiler.name, + self.pkg.spec.compiler.version, ) @property def cache_path(self): - return os.path.join(self.stage.source_path, self.cache_name) - - def flag_handler(self, name, flags): - if name in ("cflags", "cxxflags", "cppflags", "fflags"): - return (None, None, None) # handled in the cmake cache - return (flags, None, None) + return os.path.join(self.pkg.stage.source_path, self.cache_name) def initconfig_compiler_entries(self): # This will tell cmake to use the Spack compiler wrappers when run # through Spack, but use the underlying compiler when run outside of # Spack - spec = self.spec + spec = self.pkg.spec # Fortran compiler is optional if "FC" in os.environ: spack_fc_entry = cmake_cache_path("CMAKE_Fortran_COMPILER", os.environ["FC"]) - system_fc_entry = cmake_cache_path("CMAKE_Fortran_COMPILER", self.compiler.fc) + system_fc_entry = cmake_cache_path("CMAKE_Fortran_COMPILER", self.pkg.compiler.fc) else: spack_fc_entry = "# No Fortran compiler defined in spec" system_fc_entry = "# No Fortran compiler defined in spec" @@ -81,8 +85,8 @@ class CachedCMakePackage(CMakePackage): " " + cmake_cache_path("CMAKE_CXX_COMPILER", os.environ["CXX"]), " " + spack_fc_entry, "else()\n", - " " + cmake_cache_path("CMAKE_C_COMPILER", self.compiler.cc), - " " + cmake_cache_path("CMAKE_CXX_COMPILER", self.compiler.cxx), + " " + cmake_cache_path("CMAKE_C_COMPILER", self.pkg.compiler.cc), + " " + cmake_cache_path("CMAKE_CXX_COMPILER", self.pkg.compiler.cxx), " " + system_fc_entry, "endif()\n", ] @@ -126,7 +130,7 @@ class CachedCMakePackage(CMakePackage): return entries def initconfig_mpi_entries(self): - spec = self.spec + spec = self.pkg.spec if not spec.satisfies("^mpi"): return [] @@ -160,13 +164,13 @@ class CachedCMakePackage(CMakePackage): mpiexec = os.path.join(spec["mpi"].prefix.bin, "mpiexec") if not os.path.exists(mpiexec): - msg = "Unable to determine MPIEXEC, %s tests may fail" % self.name + msg = "Unable to determine MPIEXEC, %s tests may fail" % self.pkg.name entries.append("# {0}\n".format(msg)) tty.warn(msg) else: # starting with cmake 3.10, FindMPI expects MPIEXEC_EXECUTABLE # vs the older versions which expect MPIEXEC - if self.spec["cmake"].satisfies("@3.10:"): + if self.pkg.spec["cmake"].satisfies("@3.10:"): entries.append(cmake_cache_path("MPIEXEC_EXECUTABLE", mpiexec)) else: entries.append(cmake_cache_path("MPIEXEC", mpiexec)) @@ -180,7 +184,7 @@ class CachedCMakePackage(CMakePackage): return entries def initconfig_hardware_entries(self): - spec = self.spec + spec = self.pkg.spec entries = [ "#------------------{0}".format("-" * 60), @@ -212,7 +216,7 @@ class CachedCMakePackage(CMakePackage): "#------------------{0}".format("-" * 60), "# !!!! This is a generated file, edit at own risk !!!!", "#------------------{0}".format("-" * 60), - "# CMake executable path: {0}".format(self.spec["cmake"].command.path), + "# CMake executable path: {0}".format(self.pkg.spec["cmake"].command.path), "#------------------{0}\n".format("-" * 60), ] @@ -220,7 +224,8 @@ class CachedCMakePackage(CMakePackage): """This method is to be overwritten by the package""" return [] - def initconfig(self, spec, prefix): + @spack.builder.run_before("cmake") + def initconfig(self): cache_entries = ( self.std_initconfig_entries() + self.initconfig_compiler_entries() @@ -236,11 +241,28 @@ class CachedCMakePackage(CMakePackage): @property def std_cmake_args(self): - args = super(CachedCMakePackage, self).std_cmake_args + args = super(CachedCMakeBuilder, self).std_cmake_args args.extend(["-C", self.cache_path]) return args - @run_after("install") + @spack.builder.run_after("install") def install_cmake_cache(self): - mkdirp(self.spec.prefix.share.cmake) - install(self.cache_path, self.spec.prefix.share.cmake) + fs.mkdirp(self.pkg.spec.prefix.share.cmake) + fs.install(self.cache_path, self.pkg.spec.prefix.share.cmake) + + +class CachedCMakePackage(CMakePackage): + """Specialized class for packages built using CMake initial cache. + + This feature of CMake allows packages to increase reproducibility, + especially between Spack- and manual builds. It also allows packages to + sidestep certain parsing bugs in extremely long ``cmake`` commands, and to + avoid system limits on the length of the command line. + """ + + CMakeBuilder = CachedCMakeBuilder + + def flag_handler(self, name, flags): + if name in ("cflags", "cxxflags", "cppflags", "fflags"): + return None, None, None # handled in the cmake cache + return flags, None, None diff --git a/lib/spack/spack/build_systems/cmake.py b/lib/spack/spack/build_systems/cmake.py index 6fb75b6747..8d8bba4778 100644 --- a/lib/spack/spack/build_systems/cmake.py +++ b/lib/spack/spack/build_systems/cmake.py @@ -2,23 +2,26 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - - import inspect import os import platform import re import sys -from typing import List +from typing import List, Tuple import six +import llnl.util.filesystem as fs from llnl.util.compat import Sequence -from llnl.util.filesystem import working_dir import spack.build_environment -from spack.directives import conflicts, depends_on, variant -from spack.package_base import InstallError, PackageBase, run_after +import spack.builder +import spack.package_base +import spack.util.path +from spack.directives import build_system, depends_on, variant +from spack.multimethod import when + +from ._checks import BaseBuilder, execute_build_time_tests # Regex to extract the primary generator from the CMake generator # string. @@ -34,56 +37,141 @@ def _extract_primary_generator(generator): return primary_generator -class CMakePackage(PackageBase): +class CMakePackage(spack.package_base.PackageBase): """Specialized class for packages built using CMake For more information on the CMake build system, see: https://cmake.org/cmake/help/latest/ + """ + + #: This attribute is used in UI queries that need to know the build + #: system base class + build_system_class = "CMakePackage" - This class provides three phases that can be overridden: + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "cmake" - 1. :py:meth:`~.CMakePackage.cmake` - 2. :py:meth:`~.CMakePackage.build` - 3. :py:meth:`~.CMakePackage.install` + build_system("cmake") + + with when("build_system=cmake"): + # https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html + variant( + "build_type", + default="RelWithDebInfo", + description="CMake build type", + values=("Debug", "Release", "RelWithDebInfo", "MinSizeRel"), + ) + # CMAKE_INTERPROCEDURAL_OPTIMIZATION only exists for CMake >= 3.9 + # https://cmake.org/cmake/help/latest/variable/CMAKE_INTERPROCEDURAL_OPTIMIZATION.html + variant( + "ipo", + default=False, + when="^cmake@3.9:", + description="CMake interprocedural optimization", + ) + depends_on("cmake", type="build") + depends_on("ninja", type="build", when="platform=windows") + + def flags_to_build_system_args(self, flags): + """Return a list of all command line arguments to pass the specified + compiler flags to cmake. Note CMAKE does not have a cppflags option, + so cppflags will be added to cflags, cxxflags, and fflags to mimic the + behavior in other tools. + """ + # Has to be dynamic attribute due to caching + setattr(self, "cmake_flag_args", []) + + flag_string = "-DCMAKE_{0}_FLAGS={1}" + langs = {"C": "c", "CXX": "cxx", "Fortran": "f"} + + # Handle language compiler flags + for lang, pre in langs.items(): + flag = pre + "flags" + # cmake has no explicit cppflags support -> add it to all langs + lang_flags = " ".join(flags.get(flag, []) + flags.get("cppflags", [])) + if lang_flags: + self.cmake_flag_args.append(flag_string.format(lang, lang_flags)) + + # Cmake has different linker arguments for different build types. + # We specify for each of them. + if flags["ldflags"]: + ldflags = " ".join(flags["ldflags"]) + ld_string = "-DCMAKE_{0}_LINKER_FLAGS={1}" + # cmake has separate linker arguments for types of builds. + for type in ["EXE", "MODULE", "SHARED", "STATIC"]: + self.cmake_flag_args.append(ld_string.format(type, ldflags)) + + # CMake has libs options separated by language. Apply ours to each. + if flags["ldlibs"]: + libs_flags = " ".join(flags["ldlibs"]) + libs_string = "-DCMAKE_{0}_STANDARD_LIBRARIES={1}" + for lang in langs: + self.cmake_flag_args.append(libs_string.format(lang, libs_flags)) + + # Legacy methods (used by too many packages to change them, + # need to forward to the builder) + def define(self, *args, **kwargs): + return self.builder.define(*args, **kwargs) + + def define_from_variant(self, *args, **kwargs): + return self.builder.define_from_variant(*args, **kwargs) + + +@spack.builder.builder("cmake") +class CMakeBuilder(BaseBuilder): + """The cmake builder encodes the default way of building software with CMake. IT + has three phases that can be overridden: + + 1. :py:meth:`~.CMakeBuilder.cmake` + 2. :py:meth:`~.CMakeBuilder.build` + 3. :py:meth:`~.CMakeBuilder.install` They all have sensible defaults and for many packages the only thing - necessary will be to override :py:meth:`~.CMakePackage.cmake_args`. + necessary will be to override :py:meth:`~.CMakeBuilder.cmake_args`. + For a finer tuning you may also override: +-----------------------------------------------+--------------------+ | **Method** | **Purpose** | +===============================================+====================+ - | :py:meth:`~.CMakePackage.root_cmakelists_dir` | Location of the | + | :py:meth:`~.CMakeBuilder.root_cmakelists_dir` | Location of the | | | root CMakeLists.txt| +-----------------------------------------------+--------------------+ - | :py:meth:`~.CMakePackage.build_directory` | Directory where to | + | :py:meth:`~.CMakeBuilder.build_directory` | Directory where to | | | build the package | +-----------------------------------------------+--------------------+ - - The generator used by CMake can be specified by providing the - generator attribute. Per + The generator used by CMake can be specified by providing the ``generator`` + attribute. Per https://cmake.org/cmake/help/git-master/manual/cmake-generators.7.html, - the format is: [<secondary-generator> - ]<primary_generator>. The - full list of primary and secondary generators supported by CMake may - be found in the documentation for the version of CMake used; - however, at this time Spack supports only the primary generators - "Unix Makefiles" and "Ninja." Spack's CMake support is agnostic with - respect to primary generators. Spack will generate a runtime error - if the generator string does not follow the prescribed format, or if + the format is: [<secondary-generator> - ]<primary_generator>. + + The full list of primary and secondary generators supported by CMake may be found + in the documentation for the version of CMake used; however, at this time Spack + supports only the primary generators "Unix Makefiles" and "Ninja." Spack's CMake + support is agnostic with respect to primary generators. Spack will generate a + runtime error if the generator string does not follow the prescribed format, or if the primary generator is not supported. """ #: Phases of a CMake package - phases = ["cmake", "build", "install"] - #: This attribute is used in UI queries that need to know the build - #: system base class - build_system_class = "CMakePackage" - - build_targets = [] # type: List[str] - install_targets = ["install"] - - build_time_test_callbacks = ["check"] + phases = ("cmake", "build", "install") + + #: Names associated with package methods in the old build-system format + legacy_methods = ("cmake_args", "check") # type: Tuple[str, ...] + + #: Names associated with package attributes in the old build-system format + legacy_attributes = ( + "generator", + "build_targets", + "install_targets", + "build_time_test_callbacks", + "archive_files", + "root_cmakelists_dir", + "std_cmake_args", + "build_dirname", + "build_directory", + ) # type: Tuple[str, ...] #: The build system generator to use. #: @@ -93,27 +181,14 @@ class CMakePackage(PackageBase): #: #: See https://cmake.org/cmake/help/latest/manual/cmake-generators.7.html #: for more information. + generator = "Ninja" if sys.platform == "win32" else "Unix Makefiles" - generator = "Unix Makefiles" - - if sys.platform == "win32": - generator = "Ninja" - depends_on("ninja") - - # https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html - variant( - "build_type", - default="RelWithDebInfo", - description="CMake build type", - values=("Debug", "Release", "RelWithDebInfo", "MinSizeRel"), - ) - - # https://cmake.org/cmake/help/latest/variable/CMAKE_INTERPROCEDURAL_OPTIMIZATION.html - variant("ipo", default=False, description="CMake interprocedural optimization") - # CMAKE_INTERPROCEDURAL_OPTIMIZATION only exists for CMake >= 3.9 - conflicts("+ipo", when="^cmake@:3.8", msg="+ipo is not supported by CMake < 3.9") - - depends_on("cmake", type="build") + #: Targets to be used during the build phase + build_targets = [] # type: List[str] + #: Targets to be used during the install phase + install_targets = ["install"] + #: Callback names for build-time test + build_time_test_callbacks = ["check"] @property def archive_files(self): @@ -126,40 +201,30 @@ class CMakePackage(PackageBase): This path is relative to the root of the extracted tarball, not to the ``build_directory``. Defaults to the current directory. - - :return: directory containing CMakeLists.txt """ - return self.stage.source_path + return self.pkg.stage.source_path @property def std_cmake_args(self): """Standard cmake arguments provided as a property for convenience of package writers - - :return: standard cmake arguments """ # standard CMake arguments - std_cmake_args = CMakePackage._std_args(self) - std_cmake_args += getattr(self, "cmake_flag_args", []) + std_cmake_args = CMakeBuilder.std_args(self.pkg, generator=self.generator) + std_cmake_args += getattr(self.pkg, "cmake_flag_args", []) return std_cmake_args @staticmethod - def _std_args(pkg): + def std_args(pkg, generator=None): """Computes the standard cmake arguments for a generic package""" - - try: - generator = pkg.generator - except AttributeError: - generator = CMakePackage.generator - - # Make sure a valid generator was chosen + generator = generator or "Unix Makefiles" valid_primary_generators = ["Unix Makefiles", "Ninja"] primary_generator = _extract_primary_generator(generator) if primary_generator not in valid_primary_generators: msg = "Invalid CMake generator: '{0}'\n".format(generator) msg += "CMakePackage currently supports the following " msg += "primary generators: '{0}'".format("', '".join(valid_primary_generators)) - raise InstallError(msg) + raise spack.package_base.InstallError(msg) try: build_type = pkg.spec.variants["build_type"].value @@ -171,7 +236,7 @@ class CMakePackage(PackageBase): except KeyError: ipo = False - define = CMakePackage.define + define = CMakeBuilder.define args = [ "-G", generator, @@ -251,7 +316,7 @@ class CMakePackage(PackageBase): of ``cmake_var``. This utility function is similar to - :meth:`~spack.build_systems.autotools.AutotoolsPackage.with_or_without`. + :meth:`~spack.build_systems.autotools.AutotoolsBuilder.with_or_without`. Examples: @@ -291,122 +356,75 @@ class CMakePackage(PackageBase): if variant is None: variant = cmake_var.lower() - if variant not in self.variants: - raise KeyError('"{0}" is not a variant of "{1}"'.format(variant, self.name)) + if variant not in self.pkg.variants: + raise KeyError('"{0}" is not a variant of "{1}"'.format(variant, self.pkg.name)) - if variant not in self.spec.variants: + if variant not in self.pkg.spec.variants: return "" - value = self.spec.variants[variant].value + value = self.pkg.spec.variants[variant].value if isinstance(value, (tuple, list)): # Sort multi-valued variants for reproducibility value = sorted(value) return self.define(cmake_var, value) - def flags_to_build_system_args(self, flags): - """Produces a list of all command line arguments to pass the specified - compiler flags to cmake. Note CMAKE does not have a cppflags option, - so cppflags will be added to cflags, cxxflags, and fflags to mimic the - behavior in other tools.""" - # Has to be dynamic attribute due to caching - setattr(self, "cmake_flag_args", []) - - flag_string = "-DCMAKE_{0}_FLAGS={1}" - langs = {"C": "c", "CXX": "cxx", "Fortran": "f"} - - # Handle language compiler flags - for lang, pre in langs.items(): - flag = pre + "flags" - # cmake has no explicit cppflags support -> add it to all langs - lang_flags = " ".join(flags.get(flag, []) + flags.get("cppflags", [])) - if lang_flags: - self.cmake_flag_args.append(flag_string.format(lang, lang_flags)) - - # Cmake has different linker arguments for different build types. - # We specify for each of them. - if flags["ldflags"]: - ldflags = " ".join(flags["ldflags"]) - ld_string = "-DCMAKE_{0}_LINKER_FLAGS={1}" - # cmake has separate linker arguments for types of builds. - for type in ["EXE", "MODULE", "SHARED", "STATIC"]: - self.cmake_flag_args.append(ld_string.format(type, ldflags)) - - # CMake has libs options separated by language. Apply ours to each. - if flags["ldlibs"]: - libs_flags = " ".join(flags["ldlibs"]) - libs_string = "-DCMAKE_{0}_STANDARD_LIBRARIES={1}" - for lang in langs: - self.cmake_flag_args.append(libs_string.format(lang, libs_flags)) - @property def build_dirname(self): - """Returns the directory name to use when building the package - - :return: name of the subdirectory for building the package - """ - return "spack-build-%s" % self.spec.dag_hash(7) + """Directory name to use when building the package.""" + return "spack-build-%s" % self.pkg.spec.dag_hash(7) @property def build_directory(self): - """Returns the directory to use when building the package - - :return: directory where to build the package - """ - return os.path.join(self.stage.path, self.build_dirname) + """Full-path to the directory to use when building the package.""" + return os.path.join(self.pkg.stage.path, self.build_dirname) def cmake_args(self): - """Produces a list containing all the arguments that must be passed to - cmake, except: + """List of all the arguments that must be passed to cmake, except: * CMAKE_INSTALL_PREFIX * CMAKE_BUILD_TYPE * BUILD_TESTING which will be set automatically. - - :return: list of arguments for cmake """ return [] - def cmake(self, spec, prefix): + def cmake(self, pkg, spec, prefix): """Runs ``cmake`` in the build directory""" options = self.std_cmake_args options += self.cmake_args() options.append(os.path.abspath(self.root_cmakelists_dir)) - with working_dir(self.build_directory, create=True): - inspect.getmodule(self).cmake(*options) + with fs.working_dir(self.build_directory, create=True): + inspect.getmodule(self.pkg).cmake(*options) - def build(self, spec, prefix): + def build(self, pkg, spec, prefix): """Make the build targets""" - with working_dir(self.build_directory): + with fs.working_dir(self.build_directory): if self.generator == "Unix Makefiles": - inspect.getmodule(self).make(*self.build_targets) + inspect.getmodule(self.pkg).make(*self.build_targets) elif self.generator == "Ninja": self.build_targets.append("-v") - inspect.getmodule(self).ninja(*self.build_targets) + inspect.getmodule(self.pkg).ninja(*self.build_targets) - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Make the install targets""" - with working_dir(self.build_directory): + with fs.working_dir(self.build_directory): if self.generator == "Unix Makefiles": - inspect.getmodule(self).make(*self.install_targets) + inspect.getmodule(self.pkg).make(*self.install_targets) elif self.generator == "Ninja": - inspect.getmodule(self).ninja(*self.install_targets) + inspect.getmodule(self.pkg).ninja(*self.install_targets) - run_after("build")(PackageBase._run_default_build_time_test_callbacks) + spack.builder.run_after("build")(execute_build_time_tests) def check(self): - """Searches the CMake-generated Makefile for the target ``test`` - and runs it if found. + """Search the CMake-generated files for the targets ``test`` and ``check``, + and runs them if found. """ - with working_dir(self.build_directory): + with fs.working_dir(self.build_directory): if self.generator == "Unix Makefiles": - self._if_make_target_execute("test", jobs_env="CTEST_PARALLEL_LEVEL") - self._if_make_target_execute("check") + self.pkg._if_make_target_execute("test", jobs_env="CTEST_PARALLEL_LEVEL") + self.pkg._if_make_target_execute("check") elif self.generator == "Ninja": - self._if_ninja_target_execute("test", jobs_env="CTEST_PARALLEL_LEVEL") - self._if_ninja_target_execute("check") - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) + self.pkg._if_ninja_target_execute("test", jobs_env="CTEST_PARALLEL_LEVEL") + self.pkg._if_ninja_target_execute("check") diff --git a/lib/spack/spack/build_systems/generic.py b/lib/spack/spack/build_systems/generic.py new file mode 100644 index 0000000000..628af6f2d4 --- /dev/null +++ b/lib/spack/spack/build_systems/generic.py @@ -0,0 +1,44 @@ +# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other +# Spack Project Developers. See the top-level COPYRIGHT file for details. +# +# SPDX-License-Identifier: (Apache-2.0 OR MIT) +from typing import Tuple + +import spack.builder +import spack.directives +import spack.package_base + +from ._checks import BaseBuilder, apply_macos_rpath_fixups + + +class Package(spack.package_base.PackageBase): + """General purpose class with a single ``install`` phase that needs to be + coded by packagers. + """ + + #: This attribute is used in UI queries that require to know which + #: build-system class we are using + build_system_class = "Package" + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "generic" + + spack.directives.build_system("generic") + + +@spack.builder.builder("generic") +class GenericBuilder(BaseBuilder): + """A builder for a generic build system, that require packagers + to implement an "install" phase. + """ + + #: A generic package has only the "install" phase + phases = ("install",) + + #: Names associated with package methods in the old build-system format + legacy_methods = () # type: Tuple[str, ...] + + #: Names associated with package attributes in the old build-system format + legacy_attributes = ("archive_files",) # type: Tuple[str, ...] + + # On macOS, force rpaths for shared library IDs and remove duplicate rpaths + spack.builder.run_after("install", when="platform=darwin")(apply_macos_rpath_fixups) diff --git a/lib/spack/spack/build_systems/intel.py b/lib/spack/spack/build_systems/intel.py index 133b5030de..4abed70f21 100644 --- a/lib/spack/spack/build_systems/intel.py +++ b/lib/spack/spack/build_systems/intel.py @@ -2,8 +2,6 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - - import glob import inspect import os @@ -26,12 +24,14 @@ from llnl.util.filesystem import ( import spack.error from spack.build_environment import dso_suffix -from spack.package_base import InstallError, PackageBase, run_after +from spack.package_base import InstallError from spack.util.environment import EnvironmentModifications from spack.util.executable import Executable from spack.util.prefix import Prefix from spack.version import Version, ver +from .generic import Package + # A couple of utility functions that might be useful in general. If so, they # should really be defined elsewhere, unless deemed heretical. # (Or na"ive on my part). @@ -86,7 +86,7 @@ def _expand_fields(s): return s -class IntelPackage(PackageBase): +class IntelPackage(Package): """Specialized class for licensed Intel software. This class provides two phases that can be overridden: @@ -99,9 +99,6 @@ class IntelPackage(PackageBase): to set the appropriate environment variables. """ - #: Phases of an Intel package - phases = ["configure", "install"] - #: This attribute is used in UI queries that need to know the build #: system base class build_system_class = "IntelPackage" @@ -1184,12 +1181,13 @@ class IntelPackage(PackageBase): debug_print(license_type) return license_type - def configure(self, spec, prefix): + @spack.builder.run_before("install") + def configure(self): """Generates the silent.cfg file to pass to installer.sh. See https://software.intel.com/en-us/articles/configuration-file-format """ - + prefix = self.prefix # Both tokens AND values of the configuration file are validated during # the run of the underlying binary installer. Any unknown token or # unacceptable value will cause that installer to fail. Notably, this @@ -1270,7 +1268,7 @@ class IntelPackage(PackageBase): for f in glob.glob("%s/intel*log" % tmpdir): install(f, dst) - @run_after("install") + @spack.builder.run_after("install") def validate_install(self): # Sometimes the installer exits with an error but doesn't pass a # non-zero exit code to spack. Check for the existence of a 'bin' @@ -1278,7 +1276,7 @@ class IntelPackage(PackageBase): if not os.path.exists(self.prefix.bin): raise InstallError("The installer has failed to install anything.") - @run_after("install") + @spack.builder.run_after("install") def configure_rpath(self): if "+rpath" not in self.spec: return @@ -1296,7 +1294,7 @@ class IntelPackage(PackageBase): with open(compiler_cfg, "w") as fh: fh.write("-Xlinker -rpath={0}\n".format(compilers_lib_dir)) - @run_after("install") + @spack.builder.run_after("install") def configure_auto_dispatch(self): if self._has_compilers: if "auto_dispatch=none" in self.spec: @@ -1320,7 +1318,7 @@ class IntelPackage(PackageBase): with open(compiler_cfg, "a") as fh: fh.write("-ax{0}\n".format(",".join(ad))) - @run_after("install") + @spack.builder.run_after("install") def filter_compiler_wrappers(self): if ("+mpi" in self.spec or self.provides("mpi")) and "~newdtags" in self.spec: bin_dir = self.component_bin_dir("mpi") @@ -1328,7 +1326,7 @@ class IntelPackage(PackageBase): f = os.path.join(bin_dir, f) filter_file("-Xlinker --enable-new-dtags", " ", f, string=True) - @run_after("install") + @spack.builder.run_after("install") def uninstall_ism(self): # The "Intel(R) Software Improvement Program" [ahem] gets installed, # apparently regardless of PHONEHOME_SEND_USAGE_DATA. @@ -1360,7 +1358,7 @@ class IntelPackage(PackageBase): debug_print(d) return d - @run_after("install") + @spack.builder.run_after("install") def modify_LLVMgold_rpath(self): """Add libimf.so and other required libraries to the RUNPATH of LLVMgold.so. @@ -1391,6 +1389,3 @@ class IntelPackage(PackageBase): ] ) patchelf("--set-rpath", rpath, lib) - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) diff --git a/lib/spack/spack/build_systems/lua.py b/lib/spack/spack/build_systems/lua.py index c0d4321097..48fa106881 100644 --- a/lib/spack/spack/build_systems/lua.py +++ b/lib/spack/spack/build_systems/lua.py @@ -2,59 +2,79 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - - import os from llnl.util.filesystem import find -from spack.directives import depends_on, extends +import spack.builder +import spack.package_base +import spack.util.executable +from spack.directives import build_system, depends_on, extends from spack.multimethod import when -from spack.package_base import PackageBase -from spack.util.executable import Executable -class LuaPackage(PackageBase): +class LuaPackage(spack.package_base.PackageBase): """Specialized class for lua packages""" - phases = ["unpack", "generate_luarocks_config", "preprocess", "install"] #: This attribute is used in UI queries that need to know the build #: system base class build_system_class = "LuaPackage" list_depth = 1 # LuaRocks requires at least one level of spidering to find versions - depends_on("lua-lang") - extends("lua", when="^lua") - with when("^lua-luajit"): - extends("lua-luajit") - depends_on("luajit") - depends_on("lua-luajit+lualinks") - with when("^lua-luajit-openresty"): - extends("lua-luajit-openresty") - depends_on("luajit") - depends_on("lua-luajit-openresty+lualinks") - - def unpack(self, spec, prefix): - if os.path.splitext(self.stage.archive_file)[1] == ".rock": - directory = self.luarocks("unpack", self.stage.archive_file, output=str) + + build_system("lua") + + with when("build_system=lua"): + depends_on("lua-lang") + extends("lua", when="^lua") + with when("^lua-luajit"): + extends("lua-luajit") + depends_on("luajit") + depends_on("lua-luajit+lualinks") + with when("^lua-luajit-openresty"): + extends("lua-luajit-openresty") + depends_on("luajit") + depends_on("lua-luajit-openresty+lualinks") + + @property + def lua(self): + return spack.util.executable.Executable(self.spec["lua-lang"].prefix.bin.lua) + + @property + def luarocks(self): + lr = spack.util.executable.Executable(self.spec["lua-lang"].prefix.bin.luarocks) + return lr + + +@spack.builder.builder("lua") +class LuaBuilder(spack.builder.Builder): + phases = ("unpack", "generate_luarocks_config", "preprocess", "install") + + #: Names associated with package methods in the old build-system format + legacy_methods = ("luarocks_args",) + + #: Names associated with package attributes in the old build-system format + legacy_attributes = () + + def unpack(self, pkg, spec, prefix): + if os.path.splitext(pkg.stage.archive_file)[1] == ".rock": + directory = pkg.luarocks("unpack", pkg.stage.archive_file, output=str) dirlines = directory.split("\n") # TODO: figure out how to scope this better os.chdir(dirlines[2]) - def _generate_tree_line(self, name, prefix): + @staticmethod + def _generate_tree_line(name, prefix): return """{{ name = "{name}", root = "{prefix}" }};""".format( name=name, prefix=prefix, ) - def _luarocks_config_path(self): - return os.path.join(self.stage.source_path, "spack_luarocks.lua") - - def generate_luarocks_config(self, spec, prefix): - spec = self.spec + def generate_luarocks_config(self, pkg, spec, prefix): + spec = self.pkg.spec table_entries = [] for d in spec.traverse(deptypes=("build", "run"), deptype_query="run"): - if d.package.extends(self.extendee_spec): + if d.package.extends(self.pkg.extendee_spec): table_entries.append(self._generate_tree_line(d.name, d.prefix)) path = self._luarocks_config_path() @@ -71,30 +91,24 @@ class LuaPackage(PackageBase): ) return path - def setup_build_environment(self, env): - env.set("LUAROCKS_CONFIG", self._luarocks_config_path()) - - def preprocess(self, spec, prefix): + def preprocess(self, pkg, spec, prefix): """Override this to preprocess source before building with luarocks""" pass - @property - def lua(self): - return Executable(self.spec["lua-lang"].prefix.bin.lua) - - @property - def luarocks(self): - lr = Executable(self.spec["lua-lang"].prefix.bin.luarocks) - return lr - def luarocks_args(self): return [] - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): rock = "." specs = find(".", "*.rockspec", recursive=False) if specs: rock = specs[0] rocks_args = self.luarocks_args() rocks_args.append(rock) - self.luarocks("--tree=" + prefix, "make", *rocks_args) + self.pkg.luarocks("--tree=" + prefix, "make", *rocks_args) + + def _luarocks_config_path(self): + return os.path.join(self.pkg.stage.source_path, "spack_luarocks.lua") + + def setup_build_environment(self, env): + env.set("LUAROCKS_CONFIG", self._luarocks_config_path()) diff --git a/lib/spack/spack/build_systems/makefile.py b/lib/spack/spack/build_systems/makefile.py index e2bb8c0c26..b826144258 100644 --- a/lib/spack/spack/build_systems/makefile.py +++ b/lib/spack/spack/build_systems/makefile.py @@ -2,62 +2,85 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - - import inspect from typing import List # novm -import llnl.util.tty as tty -from llnl.util.filesystem import working_dir +import llnl.util.filesystem as fs + +import spack.builder +import spack.package_base +from spack.directives import build_system, conflicts -from spack.directives import conflicts -from spack.package_base import PackageBase, run_after +from ._checks import ( + BaseBuilder, + apply_macos_rpath_fixups, + execute_build_time_tests, + execute_install_time_tests, +) -class MakefilePackage(PackageBase): - """Specialized class for packages that are built using editable Makefiles +class MakefilePackage(spack.package_base.PackageBase): + """Specialized class for packages built using a Makefiles.""" - This class provides three phases that can be overridden: + #: This attribute is used in UI queries that need to know the build + #: system base class + build_system_class = "MakefilePackage" + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "makefile" - 1. :py:meth:`~.MakefilePackage.edit` - 2. :py:meth:`~.MakefilePackage.build` - 3. :py:meth:`~.MakefilePackage.install` + build_system("makefile") + conflicts("platform=windows", when="build_system=makefile") + + +@spack.builder.builder("makefile") +class MakefileBuilder(BaseBuilder): + """The Makefile builder encodes the most common way of building software with + Makefiles. It has three phases that can be overridden, if need be: + + 1. :py:meth:`~.MakefileBuilder.edit` + 2. :py:meth:`~.MakefileBuilder.build` + 3. :py:meth:`~.MakefileBuilder.install` + + It is usually necessary to override the :py:meth:`~.MakefileBuilder.edit` + phase (which is by default a no-op), while the other two have sensible defaults. - It is usually necessary to override the :py:meth:`~.MakefilePackage.edit` - phase, while :py:meth:`~.MakefilePackage.build` and - :py:meth:`~.MakefilePackage.install` have sensible defaults. For a finer tuning you may override: +-----------------------------------------------+--------------------+ | **Method** | **Purpose** | +===============================================+====================+ - | :py:attr:`~.MakefilePackage.build_targets` | Specify ``make`` | + | :py:attr:`~.MakefileBuilder.build_targets` | Specify ``make`` | | | targets for the | | | build phase | +-----------------------------------------------+--------------------+ - | :py:attr:`~.MakefilePackage.install_targets` | Specify ``make`` | + | :py:attr:`~.MakefileBuilder.install_targets` | Specify ``make`` | | | targets for the | | | install phase | +-----------------------------------------------+--------------------+ - | :py:meth:`~.MakefilePackage.build_directory` | Directory where the| + | :py:meth:`~.MakefileBuilder.build_directory` | Directory where the| | | Makefile is located| +-----------------------------------------------+--------------------+ """ - #: Phases of a package that is built with an hand-written Makefile - phases = ["edit", "build", "install"] - #: This attribute is used in UI queries that need to know the build - #: system base class - build_system_class = "MakefilePackage" + phases = ("edit", "build", "install") + + #: Names associated with package methods in the old build-system format + legacy_methods = ("check", "installcheck") - #: Targets for ``make`` during the :py:meth:`~.MakefilePackage.build` - #: phase + #: Names associated with package attributes in the old build-system format + legacy_attributes = ( + "build_targets", + "install_targets", + "build_time_test_callbacks", + "install_time_test_callbacks", + "build_directory", + ) + + #: Targets for ``make`` during the :py:meth:`~.MakefileBuilder.build` phase build_targets = [] # type: List[str] - #: Targets for ``make`` during the :py:meth:`~.MakefilePackage.install` - #: phase + #: Targets for ``make`` during the :py:meth:`~.MakefileBuilder.install` phase install_targets = ["install"] - conflicts("platform=windows") #: Callback names for build-time test build_time_test_callbacks = ["check"] @@ -66,53 +89,39 @@ class MakefilePackage(PackageBase): @property def build_directory(self): - """Returns the directory containing the main Makefile + """Return the directory containing the main Makefile.""" + return self.pkg.stage.source_path - :return: build directory - """ - return self.stage.source_path + def edit(self, pkg, spec, prefix): + """Edit the Makefile before calling make. The default is a no-op.""" + pass - def edit(self, spec, prefix): - """Edits the Makefile before calling make. This phase cannot - be defaulted. - """ - tty.msg("Using default implementation: skipping edit phase.") + def build(self, pkg, spec, prefix): + """Run "make" on the build targets specified by the builder.""" + with fs.working_dir(self.build_directory): + inspect.getmodule(self.pkg).make(*self.build_targets) - def build(self, spec, prefix): - """Calls make, passing :py:attr:`~.MakefilePackage.build_targets` - as targets. - """ - with working_dir(self.build_directory): - inspect.getmodule(self).make(*self.build_targets) - - def install(self, spec, prefix): - """Calls make, passing :py:attr:`~.MakefilePackage.install_targets` - as targets. - """ - with working_dir(self.build_directory): - inspect.getmodule(self).make(*self.install_targets) + def install(self, pkg, spec, prefix): + """Run "make" on the install targets specified by the builder.""" + with fs.working_dir(self.build_directory): + inspect.getmodule(self.pkg).make(*self.install_targets) - run_after("build")(PackageBase._run_default_build_time_test_callbacks) + spack.builder.run_after("build")(execute_build_time_tests) def check(self): - """Searches the Makefile for targets ``test`` and ``check`` - and runs them if found. - """ - with working_dir(self.build_directory): - self._if_make_target_execute("test") - self._if_make_target_execute("check") + """Run "make" on the ``test`` and ``check`` targets, if found.""" + with fs.working_dir(self.build_directory): + self.pkg._if_make_target_execute("test") + self.pkg._if_make_target_execute("check") - run_after("install")(PackageBase._run_default_install_time_test_callbacks) + spack.builder.run_after("install")(execute_install_time_tests) def installcheck(self): """Searches the Makefile for an ``installcheck`` target and runs it if found. """ - with working_dir(self.build_directory): - self._if_make_target_execute("installcheck") - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) + with fs.working_dir(self.build_directory): + self.pkg._if_make_target_execute("installcheck") # On macOS, force rpaths for shared library IDs and remove duplicate rpaths - run_after("install")(PackageBase.apply_macos_rpath_fixups) + spack.builder.run_after("install", when="platform=darwin")(apply_macos_rpath_fixups) diff --git a/lib/spack/spack/build_systems/maven.py b/lib/spack/spack/build_systems/maven.py index 1ff1882e13..66680b5b6c 100644 --- a/lib/spack/spack/build_systems/maven.py +++ b/lib/spack/spack/build_systems/maven.py @@ -2,60 +2,73 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) +import llnl.util.filesystem as fs - -from llnl.util.filesystem import install_tree, working_dir - -from spack.directives import depends_on -from spack.package_base import PackageBase, run_after +import spack.builder +import spack.package_base +from spack.directives import build_system, depends_on +from spack.multimethod import when from spack.util.executable import which +from ._checks import BaseBuilder + -class MavenPackage(PackageBase): +class MavenPackage(spack.package_base.PackageBase): """Specialized class for packages that are built using the Maven build system. See https://maven.apache.org/index.html for more information. - - This class provides the following phases that can be overridden: - - * build - * install """ - # Default phases - phases = ["build", "install"] - # To be used in UI queries that require to know which # build-system class we are using build_system_class = "MavenPackage" - depends_on("java", type=("build", "run")) - depends_on("maven", type="build") + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "maven" + + build_system("maven") + + with when("build_system=maven"): + depends_on("java", type=("build", "run")) + depends_on("maven", type="build") + + +@spack.builder.builder("maven") +class MavenBuilder(BaseBuilder): + """The Maven builder encodes the default way to build software with Maven. + It has two phases that can be overridden, if need be: + + 1. :py:meth:`~.MavenBuilder.build` + 2. :py:meth:`~.MavenBuilder.install` + """ + + phases = ("build", "install") + + #: Names associated with package methods in the old build-system format + legacy_methods = ("build_args",) + + #: Names associated with package attributes in the old build-system format + legacy_attributes = ("build_directory",) @property def build_directory(self): """The directory containing the ``pom.xml`` file.""" - return self.stage.source_path + return self.pkg.stage.source_path def build_args(self): """List of args to pass to build phase.""" return [] - def build(self, spec, prefix): + def build(self, pkg, spec, prefix): """Compile code and package into a JAR file.""" - - with working_dir(self.build_directory): + with fs.working_dir(self.build_directory): mvn = which("mvn") - if self.run_tests: + if self.pkg.run_tests: mvn("verify", *self.build_args()) else: mvn("package", "-DskipTests", *self.build_args()) - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Copy to installation prefix.""" - - with working_dir(self.build_directory): - install_tree(".", prefix) - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) + with fs.working_dir(self.build_directory): + fs.install_tree(".", prefix) diff --git a/lib/spack/spack/build_systems/meson.py b/lib/spack/spack/build_systems/meson.py index cafd613808..6ab760cfb9 100644 --- a/lib/spack/spack/build_systems/meson.py +++ b/lib/spack/spack/build_systems/meson.py @@ -2,76 +2,104 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - - import inspect import os from typing import List # novm -from llnl.util.filesystem import working_dir +import llnl.util.filesystem as fs -from spack.directives import depends_on, variant -from spack.package_base import PackageBase, run_after +import spack.builder +import spack.package_base +from spack.directives import build_system, depends_on, variant +from spack.multimethod import when +from ._checks import BaseBuilder, execute_build_time_tests -class MesonPackage(PackageBase): - """Specialized class for packages built using Meson - For more information on the Meson build system, see: - https://mesonbuild.com/ +class MesonPackage(spack.package_base.PackageBase): + """Specialized class for packages built using Meson. For more information + on the Meson build system, see https://mesonbuild.com/ + """ - This class provides three phases that can be overridden: + #: This attribute is used in UI queries that need to know the build + #: system base class + build_system_class = "MesonPackage" - 1. :py:meth:`~.MesonPackage.meson` - 2. :py:meth:`~.MesonPackage.build` - 3. :py:meth:`~.MesonPackage.install` + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "meson" + + build_system("meson") + + with when("build_system=meson"): + variant( + "buildtype", + default="debugoptimized", + description="Meson build type", + values=("plain", "debug", "debugoptimized", "release", "minsize"), + ) + variant( + "default_library", + default="shared", + values=("shared", "static"), + multi=True, + description="Build shared libs, static libs or both", + ) + variant("strip", default=False, description="Strip targets on install") + depends_on("meson", type="build") + depends_on("ninja", type="build") + + def flags_to_build_system_args(self, flags): + """Produces a list of all command line arguments to pass the specified + compiler flags to meson.""" + # Has to be dynamic attribute due to caching + setattr(self, "meson_flag_args", []) + + +@spack.builder.builder("meson") +class MesonBuilder(BaseBuilder): + """The Meson builder encodes the default way to build software with Meson. + The builder has three phases that can be overridden, if need be: + + 1. :py:meth:`~.MesonBuilder.meson` + 2. :py:meth:`~.MesonBuilder.build` + 3. :py:meth:`~.MesonBuilder.install` They all have sensible defaults and for many packages the only thing - necessary will be to override :py:meth:`~.MesonPackage.meson_args`. + necessary will be to override :py:meth:`~.MesonBuilder.meson_args`. + For a finer tuning you may also override: +-----------------------------------------------+--------------------+ | **Method** | **Purpose** | +===============================================+====================+ - | :py:meth:`~.MesonPackage.root_mesonlists_dir` | Location of the | + | :py:meth:`~.MesonBuilder.root_mesonlists_dir` | Location of the | | | root MesonLists.txt| +-----------------------------------------------+--------------------+ - | :py:meth:`~.MesonPackage.build_directory` | Directory where to | + | :py:meth:`~.MesonBuilder.build_directory` | Directory where to | | | build the package | +-----------------------------------------------+--------------------+ + """ + phases = ("meson", "build", "install") - """ + #: Names associated with package methods in the old build-system format + legacy_methods = ("meson_args", "check") - #: Phases of a Meson package - phases = ["meson", "build", "install"] - #: This attribute is used in UI queries that need to know the build - #: system base class - build_system_class = "MesonPackage" + #: Names associated with package attributes in the old build-system format + legacy_attributes = ( + "build_targets", + "install_targets", + "build_time_test_callbacks", + "root_mesonlists_dir", + "std_meson_args", + "build_directory", + ) build_targets = [] # type: List[str] install_targets = ["install"] build_time_test_callbacks = ["check"] - variant( - "buildtype", - default="debugoptimized", - description="Meson build type", - values=("plain", "debug", "debugoptimized", "release", "minsize"), - ) - variant( - "default_library", - default="shared", - values=("shared", "static"), - multi=True, - description="Build shared libs, static libs or both", - ) - variant("strip", default=False, description="Strip targets on install") - - depends_on("meson", type="build") - depends_on("ninja", type="build") - @property def archive_files(self): """Files to archive for packages based on Meson""" @@ -79,31 +107,26 @@ class MesonPackage(PackageBase): @property def root_mesonlists_dir(self): - """The relative path to the directory containing meson.build + """Relative path to the directory containing meson.build This path is relative to the root of the extracted tarball, not to the ``build_directory``. Defaults to the current directory. - - :return: directory containing meson.build """ - return self.stage.source_path + return self.pkg.stage.source_path @property def std_meson_args(self): - """Standard meson arguments provided as a property for - convenience of package writers - - :return: standard meson arguments + """Standard meson arguments provided as a property for convenience + of package writers. """ # standard Meson arguments - std_meson_args = MesonPackage._std_args(self) + std_meson_args = MesonBuilder.std_args(self.pkg) std_meson_args += getattr(self, "meson_flag_args", []) return std_meson_args @staticmethod - def _std_args(pkg): - """Computes the standard meson arguments for a generic package""" - + def std_args(pkg): + """Standard meson arguments for a generic package.""" try: build_type = pkg.spec.variants["buildtype"].value except KeyError: @@ -132,31 +155,18 @@ class MesonPackage(PackageBase): return args - def flags_to_build_system_args(self, flags): - """Produces a list of all command line arguments to pass the specified - compiler flags to meson.""" - # Has to be dynamic attribute due to caching - setattr(self, "meson_flag_args", []) - @property def build_dirname(self): - """Returns the directory name to use when building the package - - :return: name of the subdirectory for building the package - """ - return "spack-build-%s" % self.spec.dag_hash(7) + """Returns the directory name to use when building the package.""" + return "spack-build-{}".format(self.spec.dag_hash(7)) @property def build_directory(self): - """Returns the directory to use when building the package - - :return: directory where to build the package - """ - return os.path.join(self.stage.path, self.build_dirname) + """Directory to use when building the package.""" + return os.path.join(self.pkg.stage.path, self.build_dirname) def meson_args(self): - """Produces a list containing all the arguments that must be passed to - meson, except: + """List of arguments that must be passed to meson, except: * ``--prefix`` * ``--libdir`` @@ -165,40 +175,33 @@ class MesonPackage(PackageBase): * ``--default_library`` which will be set automatically. - - :return: list of arguments for meson """ return [] - def meson(self, spec, prefix): - """Runs ``meson`` in the build directory""" + def meson(self, pkg, spec, prefix): + """Run ``meson`` in the build directory""" options = [os.path.abspath(self.root_mesonlists_dir)] options += self.std_meson_args options += self.meson_args() - with working_dir(self.build_directory, create=True): - inspect.getmodule(self).meson(*options) + with fs.working_dir(self.build_directory, create=True): + inspect.getmodule(self.pkg).meson(*options) - def build(self, spec, prefix): + def build(self, pkg, spec, prefix): """Make the build targets""" options = ["-v"] options += self.build_targets - with working_dir(self.build_directory): - inspect.getmodule(self).ninja(*options) + with fs.working_dir(self.build_directory): + inspect.getmodule(self.pkg).ninja(*options) - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Make the install targets""" - with working_dir(self.build_directory): - inspect.getmodule(self).ninja(*self.install_targets) + with fs.working_dir(self.build_directory): + inspect.getmodule(self.pkg).ninja(*self.install_targets) - run_after("build")(PackageBase._run_default_build_time_test_callbacks) + spack.builder.run_after("build")(execute_build_time_tests) def check(self): - """Searches the Meson-generated file for the target ``test`` - and runs it if found. - """ - with working_dir(self.build_directory): + """Search Meson-generated files for the target ``test`` and run it if found.""" + with fs.working_dir(self.build_directory): self._if_ninja_target_execute("test") self._if_ninja_target_execute("check") - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) diff --git a/lib/spack/spack/build_systems/nmake.py b/lib/spack/spack/build_systems/nmake.py new file mode 100644 index 0000000000..bf36895160 --- /dev/null +++ b/lib/spack/spack/build_systems/nmake.py @@ -0,0 +1,102 @@ +# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other +# Spack Project Developers. See the top-level COPYRIGHT file for details. +# +# SPDX-License-Identifier: (Apache-2.0 OR MIT) +import inspect +from typing import List # novm + +import llnl.util.filesystem as fs + +import spack.builder +import spack.package_base +from spack.directives import build_system, conflicts + +from ._checks import BaseBuilder + + +class NMakePackage(spack.package_base.PackageBase): + """Specialized class for packages built using a Makefiles.""" + + #: This attribute is used in UI queries that need to know the build + #: system base class + build_system_class = "NmakePackage" + + build_system("nmake") + conflicts("platform=linux", when="build_system=nmake") + conflicts("platform=darwin", when="build_system=nmake") + conflicts("platform=cray", when="build_system=nmake") + + +@spack.builder.builder("nmake") +class NMakeBuilder(BaseBuilder): + """The NMake builder encodes the most common way of building software with + NMake on Windows. It has three phases that can be overridden, if need be: + + 1. :py:meth:`~.NMakeBuilder.edit` + 2. :py:meth:`~.NMakeBuilder.build` + 3. :py:meth:`~.NMakeBuilder.install` + + It is usually necessary to override the :py:meth:`~.NMakeBuilder.edit` + phase (which is by default a no-op), while the other two have sensible defaults. + + For a finer tuning you may override: + + +--------------------------------------------+--------------------+ + | **Method** | **Purpose** | + +============================================+====================+ + | :py:attr:`~.NMakeBuilder.build_targets` | Specify ``nmake`` | + | | targets for the | + | | build phase | + +--------------------------------------------+--------------------+ + | :py:attr:`~.NMakeBuilder.install_targets` | Specify ``nmake`` | + | | targets for the | + | | install phase | + +--------------------------------------------+--------------------+ + | :py:meth:`~.NMakeBuilder.build_directory` | Directory where the| + | | Makefile is located| + +--------------------------------------------+--------------------+ + """ + + phases = ("edit", "build", "install") + + #: Names associated with package methods in the old build-system format + legacy_methods = ("check", "installcheck") + + #: Names associated with package attributes in the old build-system format + legacy_attributes = ( + "build_targets", + "install_targets", + "build_time_test_callbacks", + "install_time_test_callbacks", + "build_directory", + ) + + #: Targets for ``make`` during the :py:meth:`~.NMakeBuilder.build` phase + build_targets = [] # type: List[str] + #: Targets for ``make`` during the :py:meth:`~.NMakeBuilder.install` phase + install_targets = ["install"] + + #: Callback names for build-time test + build_time_test_callbacks = ["check"] + + #: Callback names for install-time test + install_time_test_callbacks = ["installcheck"] + + @property + def build_directory(self): + """Return the directory containing the main Makefile.""" + return self.pkg.stage.source_path + + def edit(self, pkg, spec, prefix): + """Edit the Makefile before calling make. The default is a no-op.""" + pass + + def build(self, pkg, spec, prefix): + """Run "make" on the build targets specified by the builder.""" + with fs.working_dir(self.build_directory): + inspect.getmodule(self.pkg).nmake(*self.build_targets) + + def install(self, pkg, spec, prefix): + """Run "make" on the install targets specified by the builder.""" + with fs.working_dir(self.build_directory): + inspect.getmodule(self.pkg).nmake(*self.install_targets) diff --git a/lib/spack/spack/build_systems/octave.py b/lib/spack/spack/build_systems/octave.py index 9916c319b0..5b2456f098 100644 --- a/lib/spack/spack/build_systems/octave.py +++ b/lib/spack/spack/build_systems/octave.py @@ -2,51 +2,62 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - import inspect -from spack.directives import extends -from spack.package_base import PackageBase, run_after +import spack.builder +import spack.package_base +from spack.directives import build_system, extends +from spack.multimethod import when + +from ._checks import BaseBuilder -class OctavePackage(PackageBase): +class OctavePackage(spack.package_base.PackageBase): """Specialized class for Octave packages. See https://www.gnu.org/software/octave/doc/v4.2.0/Installing-and-Removing-Packages.html for more information. - - This class provides the following phases that can be overridden: - - 1. :py:meth:`~.OctavePackage.install` - """ - # Default phases - phases = ["install"] - # To be used in UI queries that require to know which # build-system class we are using build_system_class = "OctavePackage" + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "octave" - extends("octave") + build_system("octave") + + with when("build_system=octave"): + extends("octave") - def setup_build_environment(self, env): - # octave does not like those environment variables to be set: - env.unset("CC") - env.unset("CXX") - env.unset("FC") - def install(self, spec, prefix): +@spack.builder.builder("octave") +class OctaveBuilder(BaseBuilder): + """The octave builder provides the following phases that can be overridden: + + 1. :py:meth:`~.OctaveBuilder.install` + """ + + phases = ("install",) + + #: Names associated with package methods in the old build-system format + legacy_methods = () + + #: Names associated with package attributes in the old build-system format + legacy_attributes = () + + def install(self, pkg, spec, prefix): """Install the package from the archive file""" - inspect.getmodule(self).octave( + inspect.getmodule(self.pkg).octave( "--quiet", "--norc", "--built-in-docstrings-file=/dev/null", "--texi-macros-file=/dev/null", "--eval", - "pkg prefix %s; pkg install %s" % (prefix, self.stage.archive_file), + "pkg prefix %s; pkg install %s" % (prefix, self.pkg.stage.archive_file), ) - # Testing - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) + def setup_build_environment(self, env): + # octave does not like those environment variables to be set: + env.unset("CC") + env.unset("CXX") + env.unset("FC") diff --git a/lib/spack/spack/build_systems/oneapi.py b/lib/spack/spack/build_systems/oneapi.py index 669c66fe8f..1cb79b9901 100644 --- a/lib/spack/spack/build_systems/oneapi.py +++ b/lib/spack/spack/build_systems/oneapi.py @@ -2,11 +2,7 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - -"""Common utilities for managing intel oneapi packages. - -""" - +"""Common utilities for managing intel oneapi packages.""" import getpass import platform import shutil @@ -14,18 +10,17 @@ from os.path import basename, dirname, isdir from llnl.util.filesystem import find_headers, find_libraries, join_path -from spack.package_base import Package from spack.util.environment import EnvironmentModifications from spack.util.executable import Executable +from .generic import Package + class IntelOneApiPackage(Package): """Base class for Intel oneAPI packages.""" homepage = "https://software.intel.com/oneapi" - phases = ["install"] - # oneAPI license does not allow mirroring outside of the # organization (e.g. University/Company). redistribute_source = False diff --git a/lib/spack/spack/build_systems/perl.py b/lib/spack/spack/build_systems/perl.py index 1f354beece..a100b89bfd 100644 --- a/lib/spack/spack/build_systems/perl.py +++ b/lib/spack/spack/build_systems/perl.py @@ -2,73 +2,87 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - - import inspect import os from llnl.util.filesystem import filter_file -from spack.directives import extends -from spack.package_base import PackageBase, run_after +import spack.builder +import spack.package_base +from spack.directives import build_system, extends +from spack.package_base import PackageBase from spack.util.executable import Executable +from ._checks import BaseBuilder, execute_build_time_tests + class PerlPackage(PackageBase): - """Specialized class for packages that are built using Perl. + """Specialized class for packages that are built using Perl.""" + + #: This attribute is used in UI queries that need to know the build + #: system base class + build_system_class = "PerlPackage" + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "perl" + + build_system("perl") + + extends("perl", when="build_system=perl") - This class provides four phases that can be overridden if required: - 1. :py:meth:`~.PerlPackage.configure` - 2. :py:meth:`~.PerlPackage.build` - 3. :py:meth:`~.PerlPackage.check` - 4. :py:meth:`~.PerlPackage.install` +@spack.builder.builder("perl") +class PerlBuilder(BaseBuilder): + """The perl builder provides four phases that can be overridden, if required: + + 1. :py:meth:`~.PerlBuilder.configure` + 2. :py:meth:`~.PerlBuilder.build` + 3. :py:meth:`~.PerlBuilder.check` + 4. :py:meth:`~.PerlBuilder.install` The default methods use, in order of preference: (1) Makefile.PL, (2) Build.PL. - Some packages may need to override - :py:meth:`~.PerlPackage.configure_args`, - which produces a list of arguments for - :py:meth:`~.PerlPackage.configure`. + Some packages may need to override :py:meth:`~.PerlBuilder.configure_args`, + which produces a list of arguments for :py:meth:`~.PerlBuilder.configure`. + Arguments should not include the installation base directory. """ #: Phases of a Perl package - phases = ["configure", "build", "install"] + phases = ("configure", "build", "install") - #: This attribute is used in UI queries that need to know the build - #: system base class - build_system_class = "PerlPackage" + #: Names associated with package methods in the old build-system format + legacy_methods = ("configure_args", "check") + + #: Names associated with package attributes in the old build-system format + legacy_attributes = () #: Callback names for build-time test build_time_test_callbacks = ["check"] - extends("perl") - def configure_args(self): - """Produces a list containing the arguments that must be passed to - :py:meth:`~.PerlPackage.configure`. Arguments should not include - the installation base directory, which is prepended automatically. + """List of arguments passed to :py:meth:`~.PerlBuilder.configure`. - :return: list of arguments for Makefile.PL or Build.PL + Arguments should not include the installation base directory, which + is prepended automatically. """ return [] - def configure(self, spec, prefix): - """Runs Makefile.PL or Build.PL with arguments consisting of + def configure(self, pkg, spec, prefix): + """Run Makefile.PL or Build.PL with arguments consisting of an appropriate installation base directory followed by the - list returned by :py:meth:`~.PerlPackage.configure_args`. + list returned by :py:meth:`~.PerlBuilder.configure_args`. - :raise RuntimeError: if neither Makefile.PL or Build.PL exist + Raises: + RuntimeError: if neither Makefile.PL nor Build.PL exist """ if os.path.isfile("Makefile.PL"): self.build_method = "Makefile.PL" - self.build_executable = inspect.getmodule(self).make + self.build_executable = inspect.getmodule(self.pkg).make elif os.path.isfile("Build.PL"): self.build_method = "Build.PL" - self.build_executable = Executable(os.path.join(self.stage.source_path, "Build")) + self.build_executable = Executable(os.path.join(self.pkg.stage.source_path, "Build")) else: raise RuntimeError("Unknown build_method for perl package") @@ -78,33 +92,30 @@ class PerlPackage(PackageBase): options = ["Build.PL", "--install_base", prefix] options += self.configure_args() - inspect.getmodule(self).perl(*options) + inspect.getmodule(self.pkg).perl(*options) # It is possible that the shebang in the Build script that is created from # Build.PL may be too long causing the build to fail. Patching the shebang # does not happen until after install so set '/usr/bin/env perl' here in # the Build script. - @run_after("configure") + @spack.builder.run_after("configure") def fix_shebang(self): if self.build_method == "Build.PL": pattern = "#!{0}".format(self.spec["perl"].command.path) repl = "#!/usr/bin/env perl" filter_file(pattern, repl, "Build", backup=False) - def build(self, spec, prefix): + def build(self, pkg, spec, prefix): """Builds a Perl package.""" self.build_executable() # Ensure that tests run after build (if requested): - run_after("build")(PackageBase._run_default_build_time_test_callbacks) + spack.builder.run_after("build")(execute_build_time_tests) def check(self): """Runs built-in tests of a Perl package.""" self.build_executable("test") - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Installs a Perl package.""" self.build_executable("install") - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) diff --git a/lib/spack/spack/build_systems/python.py b/lib/spack/spack/build_systems/python.py index d1cecdac63..0f84ac7f8a 100644 --- a/lib/spack/spack/build_systems/python.py +++ b/lib/spack/spack/build_systems/python.py @@ -8,93 +8,22 @@ import re import shutil from typing import Optional +import llnl.util.filesystem as fs +import llnl.util.lang as lang import llnl.util.tty as tty -from llnl.util.filesystem import ( - filter_file, - find, - find_all_headers, - find_libraries, - is_nonsymlink_exe_with_shebang, - path_contains_subdirectory, - same_path, - working_dir, -) -from llnl.util.lang import classproperty, match_predicate - -from spack.directives import depends_on, extends + +import spack.builder +import spack.multimethod +import spack.package_base +from spack.directives import build_system, depends_on, extends from spack.error import NoHeadersError, NoLibrariesError, SpecError -from spack.package_base import PackageBase, run_after from spack.version import Version +from ._checks import BaseBuilder, execute_install_time_tests -class PythonPackage(PackageBase): - """Specialized class for packages that are built using pip.""" - - #: Package name, version, and extension on PyPI - pypi = None # type: Optional[str] - - maintainers = ["adamjstewart", "pradyunsg"] - - # Default phases - phases = ["install"] - - # To be used in UI queries that require to know which - # build-system class we are using - build_system_class = "PythonPackage" - - #: Callback names for install-time test - install_time_test_callbacks = ["test"] - - extends("python") - depends_on("py-pip", type="build") - # FIXME: technically wheel is only needed when building from source, not when - # installing a downloaded wheel, but I don't want to add wheel as a dep to every - # package manually - depends_on("py-wheel", type="build") - - py_namespace = None # type: Optional[str] - - @staticmethod - def _std_args(cls): - return [ - # Verbose - "-vvv", - # Disable prompting for input - "--no-input", - # Disable the cache - "--no-cache-dir", - # Don't check to see if pip is up-to-date - "--disable-pip-version-check", - # Install packages - "install", - # Don't install package dependencies - "--no-deps", - # Overwrite existing packages - "--ignore-installed", - # Use env vars like PYTHONPATH - "--no-build-isolation", - # Don't warn that prefix.bin is not in PATH - "--no-warn-script-location", - # Ignore the PyPI package index - "--no-index", - ] - - @classproperty - def homepage(cls): - if cls.pypi: - name = cls.pypi.split("/")[0] - return "https://pypi.org/project/" + name + "/" - - @classproperty - def url(cls): - if cls.pypi: - return "https://files.pythonhosted.org/packages/source/" + cls.pypi[0] + "/" + cls.pypi - @classproperty - def list_url(cls): - if cls.pypi: - name = cls.pypi.split("/")[0] - return "https://pypi.org/simple/" + name + "/" +class PythonExtension(spack.package_base.PackageBase): + maintainers = ["adamjstewart"] @property def import_modules(self): @@ -124,7 +53,7 @@ class PythonPackage(PackageBase): # Some Python libraries are packages: collections of modules # distributed in directories containing __init__.py files - for path in find(root, "__init__.py", recursive=True): + for path in fs.find(root, "__init__.py", recursive=True): modules.append( path.replace(root + os.sep, "", 1) .replace(os.sep + "__init__.py", "") @@ -133,7 +62,7 @@ class PythonPackage(PackageBase): # Some Python libraries are modules: individual *.py files # found in the site-packages directory - for path in find(root, "*.py", recursive=False): + for path in fs.find(root, "*.py", recursive=False): modules.append( path.replace(root + os.sep, "", 1).replace(".py", "").replace("/", ".") ) @@ -160,6 +89,208 @@ class PythonPackage(PackageBase): """ return [] + def view_file_conflicts(self, view, merge_map): + """Report all file conflicts, excepting special cases for python. + Specifically, this does not report errors for duplicate + __init__.py files for packages in the same namespace. + """ + conflicts = list(dst for src, dst in merge_map.items() if os.path.exists(dst)) + + if conflicts and self.py_namespace: + ext_map = view.extensions_layout.extension_map(self.extendee_spec) + namespaces = set(x.package.py_namespace for x in ext_map.values()) + namespace_re = r"site-packages/{0}/__init__.py".format(self.py_namespace) + find_namespace = lang.match_predicate(namespace_re) + if self.py_namespace in namespaces: + conflicts = list(x for x in conflicts if not find_namespace(x)) + + return conflicts + + def add_files_to_view(self, view, merge_map, skip_if_exists=True): + bin_dir = self.spec.prefix.bin + python_prefix = self.extendee_spec.prefix + python_is_external = self.extendee_spec.external + global_view = fs.same_path(python_prefix, view.get_projection_for_spec(self.spec)) + for src, dst in merge_map.items(): + if os.path.exists(dst): + continue + elif global_view or not fs.path_contains_subdirectory(src, bin_dir): + view.link(src, dst) + elif not os.path.islink(src): + shutil.copy2(src, dst) + is_script = fs.is_nonsymlink_exe_with_shebang(src) + if is_script and not python_is_external: + fs.filter_file( + python_prefix, + os.path.abspath(view.get_projection_for_spec(self.spec)), + dst, + ) + else: + orig_link_target = os.path.realpath(src) + new_link_target = os.path.abspath(merge_map[orig_link_target]) + view.link(new_link_target, dst) + + def remove_files_from_view(self, view, merge_map): + ignore_namespace = False + if self.py_namespace: + ext_map = view.extensions_layout.extension_map(self.extendee_spec) + remaining_namespaces = set( + spec.package.py_namespace for name, spec in ext_map.items() if name != self.name + ) + if self.py_namespace in remaining_namespaces: + namespace_init = lang.match_predicate( + r"site-packages/{0}/__init__.py".format(self.py_namespace) + ) + ignore_namespace = True + + bin_dir = self.spec.prefix.bin + global_view = self.extendee_spec.prefix == view.get_projection_for_spec(self.spec) + + to_remove = [] + for src, dst in merge_map.items(): + if ignore_namespace and namespace_init(dst): + continue + + if global_view or not fs.path_contains_subdirectory(src, bin_dir): + to_remove.append(dst) + else: + os.remove(dst) + + view.remove_files(to_remove) + + def test(self): + """Attempts to import modules of the installed package.""" + + # Make sure we are importing the installed modules, + # not the ones in the source directory + for module in self.import_modules: + self.run_test( + inspect.getmodule(self).python.path, + ["-c", "import {0}".format(module)], + purpose="checking import of {0}".format(module), + work_dir="spack-test", + ) + + +class PythonPackage(PythonExtension): + """Specialized class for packages that are built using pip.""" + + #: Package name, version, and extension on PyPI + pypi = None # type: Optional[str] + + maintainers = ["adamjstewart", "pradyunsg"] + + # To be used in UI queries that require to know which + # build-system class we are using + build_system_class = "PythonPackage" + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "python_pip" + + #: Callback names for install-time test + install_time_test_callbacks = ["test"] + + build_system("python_pip") + + with spack.multimethod.when("build_system=python_pip"): + extends("python") + depends_on("py-pip", type="build") + # FIXME: technically wheel is only needed when building from source, not when + # installing a downloaded wheel, but I don't want to add wheel as a dep to every + # package manually + depends_on("py-wheel", type="build") + + py_namespace = None # type: Optional[str] + + @lang.classproperty + def homepage(cls): + if cls.pypi: + name = cls.pypi.split("/")[0] + return "https://pypi.org/project/" + name + "/" + + @lang.classproperty + def url(cls): + if cls.pypi: + return "https://files.pythonhosted.org/packages/source/" + cls.pypi[0] + "/" + cls.pypi + + @lang.classproperty + def list_url(cls): + if cls.pypi: + name = cls.pypi.split("/")[0] + return "https://pypi.org/simple/" + name + "/" + + @property + def headers(self): + """Discover header files in platlib.""" + + # Headers may be in either location + include = self.prefix.join(self.spec["python"].package.include) + platlib = self.prefix.join(self.spec["python"].package.platlib) + headers = fs.find_all_headers(include) + fs.find_all_headers(platlib) + + if headers: + return headers + + msg = "Unable to locate {} headers in {} or {}" + raise NoHeadersError(msg.format(self.spec.name, include, platlib)) + + @property + def libs(self): + """Discover libraries in platlib.""" + + # Remove py- prefix in package name + library = "lib" + self.spec.name[3:].replace("-", "?") + root = self.prefix.join(self.spec["python"].package.platlib) + + for shared in [True, False]: + libs = fs.find_libraries(library, root, shared=shared, recursive=True) + if libs: + return libs + + msg = "Unable to recursively locate {} libraries in {}" + raise NoLibrariesError(msg.format(self.spec.name, root)) + + +@spack.builder.builder("python_pip") +class PythonPipBuilder(BaseBuilder): + phases = ("install",) + + #: Names associated with package methods in the old build-system format + legacy_methods = ("test",) + + #: Same as legacy_methods, but the signature is different + legacy_long_methods = ("install_options", "global_options", "config_settings") + + #: Names associated with package attributes in the old build-system format + legacy_attributes = ("build_directory", "install_time_test_callbacks") + + #: Callback names for install-time test + install_time_test_callbacks = ["test"] + + @staticmethod + def std_args(cls): + return [ + # Verbose + "-vvv", + # Disable prompting for input + "--no-input", + # Disable the cache + "--no-cache-dir", + # Don't check to see if pip is up-to-date + "--disable-pip-version-check", + # Install packages + "install", + # Don't install package dependencies + "--no-deps", + # Overwrite existing packages + "--ignore-installed", + # Use env vars like PYTHONPATH + "--no-build-isolation", + # Don't warn that prefix.bin is not in PATH + "--no-warn-script-location", + # Ignore the PyPI package index + "--no-index", + ] + @property def build_directory(self): """The root directory of the Python package. @@ -170,11 +301,10 @@ class PythonPackage(PackageBase): * ``setup.cfg`` * ``setup.py`` """ - return self.stage.source_path + return self.pkg.stage.source_path def config_settings(self, spec, prefix): """Configuration settings to be passed to the PEP 517 build backend. - Requires pip 22.1+, which requires Python 3.7+. Args: @@ -211,10 +341,10 @@ class PythonPackage(PackageBase): """ return [] - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Install everything from build directory.""" - args = PythonPackage._std_args(self) + ["--prefix=" + prefix] + args = PythonPipBuilder.std_args(pkg) + ["--prefix=" + prefix] for key, value in self.config_settings(spec, prefix).items(): if spec["py-pip"].version < Version("22.1"): @@ -223,137 +353,21 @@ class PythonPackage(PackageBase): "pip 22.1+. Add the following line to the package to fix this:\n\n" ' depends_on("py-pip@22.1:", type="build")'.format(spec.name) ) + args.append("--config-settings={}={}".format(key, value)) + for option in self.install_options(spec, prefix): args.append("--install-option=" + option) for option in self.global_options(spec, prefix): args.append("--global-option=" + option) - if self.stage.archive_file and self.stage.archive_file.endswith(".whl"): - args.append(self.stage.archive_file) + if pkg.stage.archive_file and pkg.stage.archive_file.endswith(".whl"): + args.append(pkg.stage.archive_file) else: args.append(".") - pip = inspect.getmodule(self).pip - with working_dir(self.build_directory): + pip = inspect.getmodule(pkg).pip + with fs.working_dir(self.build_directory): pip(*args) - @property - def headers(self): - """Discover header files in platlib.""" - - # Headers may be in either location - include = self.prefix.join(self.spec["python"].package.include) - platlib = self.prefix.join(self.spec["python"].package.platlib) - headers = find_all_headers(include) + find_all_headers(platlib) - - if headers: - return headers - - msg = "Unable to locate {} headers in {} or {}" - raise NoHeadersError(msg.format(self.spec.name, include, platlib)) - - @property - def libs(self): - """Discover libraries in platlib.""" - - # Remove py- prefix in package name - library = "lib" + self.spec.name[3:].replace("-", "?") - root = self.prefix.join(self.spec["python"].package.platlib) - - for shared in [True, False]: - libs = find_libraries(library, root, shared=shared, recursive=True) - if libs: - return libs - - msg = "Unable to recursively locate {} libraries in {}" - raise NoLibrariesError(msg.format(self.spec.name, root)) - - # Testing - - def test(self): - """Attempts to import modules of the installed package.""" - - # Make sure we are importing the installed modules, - # not the ones in the source directory - for module in self.import_modules: - self.run_test( - inspect.getmodule(self).python.path, - ["-c", "import {0}".format(module)], - purpose="checking import of {0}".format(module), - work_dir="spack-test", - ) - - run_after("install")(PackageBase._run_default_install_time_test_callbacks) - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) - - def view_file_conflicts(self, view, merge_map): - """Report all file conflicts, excepting special cases for python. - Specifically, this does not report errors for duplicate - __init__.py files for packages in the same namespace. - """ - conflicts = list(dst for src, dst in merge_map.items() if os.path.exists(dst)) - - if conflicts and self.py_namespace: - ext_map = view.extensions_layout.extension_map(self.extendee_spec) - namespaces = set(x.package.py_namespace for x in ext_map.values()) - namespace_re = r"site-packages/{0}/__init__.py".format(self.py_namespace) - find_namespace = match_predicate(namespace_re) - if self.py_namespace in namespaces: - conflicts = list(x for x in conflicts if not find_namespace(x)) - - return conflicts - - def add_files_to_view(self, view, merge_map, skip_if_exists=True): - bin_dir = self.spec.prefix.bin - python_prefix = self.extendee_spec.prefix - python_is_external = self.extendee_spec.external - global_view = same_path(python_prefix, view.get_projection_for_spec(self.spec)) - for src, dst in merge_map.items(): - if os.path.exists(dst): - continue - elif global_view or not path_contains_subdirectory(src, bin_dir): - view.link(src, dst) - elif not os.path.islink(src): - shutil.copy2(src, dst) - is_script = is_nonsymlink_exe_with_shebang(src) - if is_script and not python_is_external: - filter_file( - python_prefix, - os.path.abspath(view.get_projection_for_spec(self.spec)), - dst, - ) - else: - orig_link_target = os.path.realpath(src) - new_link_target = os.path.abspath(merge_map[orig_link_target]) - view.link(new_link_target, dst) - - def remove_files_from_view(self, view, merge_map): - ignore_namespace = False - if self.py_namespace: - ext_map = view.extensions_layout.extension_map(self.extendee_spec) - remaining_namespaces = set( - spec.package.py_namespace for name, spec in ext_map.items() if name != self.name - ) - if self.py_namespace in remaining_namespaces: - namespace_init = match_predicate( - r"site-packages/{0}/__init__.py".format(self.py_namespace) - ) - ignore_namespace = True - - bin_dir = self.spec.prefix.bin - global_view = self.extendee_spec.prefix == view.get_projection_for_spec(self.spec) - - to_remove = [] - for src, dst in merge_map.items(): - if ignore_namespace and namespace_init(dst): - continue - - if global_view or not path_contains_subdirectory(src, bin_dir): - to_remove.append(dst) - else: - os.remove(dst) - - view.remove_files(to_remove) + spack.builder.run_after("install")(execute_install_time_tests) diff --git a/lib/spack/spack/build_systems/qmake.py b/lib/spack/spack/build_systems/qmake.py index c2af684592..f18bd9812f 100644 --- a/lib/spack/spack/build_systems/qmake.py +++ b/lib/spack/spack/build_systems/qmake.py @@ -2,82 +2,85 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - - import inspect from llnl.util.filesystem import working_dir -from spack.directives import depends_on -from spack.package_base import PackageBase, run_after +import spack.builder +import spack.package_base +from spack.directives import build_system, depends_on + +from ._checks import BaseBuilder, execute_build_time_tests -class QMakePackage(PackageBase): +class QMakePackage(spack.package_base.PackageBase): """Specialized class for packages built using qmake. For more information on the qmake build system, see: http://doc.qt.io/qt-5/qmake-manual.html + """ + + #: This attribute is used in UI queries that need to know the build + #: system base class + build_system_class = "QMakePackage" + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "qmake" + + build_system("qmake") + + depends_on("qt", type="build", when="build_system=qmake") + - This class provides three phases that can be overridden: +@spack.builder.builder("qmake") +class QMakeBuilder(BaseBuilder): + """The qmake builder provides three phases that can be overridden: - 1. :py:meth:`~.QMakePackage.qmake` - 2. :py:meth:`~.QMakePackage.build` - 3. :py:meth:`~.QMakePackage.install` + 1. :py:meth:`~.QMakeBuilder.qmake` + 2. :py:meth:`~.QMakeBuilder.build` + 3. :py:meth:`~.QMakeBuilder.install` They all have sensible defaults and for many packages the only thing - necessary will be to override :py:meth:`~.QMakePackage.qmake_args`. + necessary will be to override :py:meth:`~.QMakeBuilder.qmake_args`. """ - #: Phases of a qmake package - phases = ["qmake", "build", "install"] + phases = ("qmake", "build", "install") - #: This attribute is used in UI queries that need to know the build - #: system base class - build_system_class = "QMakePackage" + #: Names associated with package methods in the old build-system format + legacy_methods = ("qmake_args", "check") + + #: Names associated with package attributes in the old build-system format + legacy_attributes = ("build_directory", "build_time_test_callbacks") #: Callback names for build-time test build_time_test_callbacks = ["check"] - depends_on("qt", type="build") - @property def build_directory(self): """The directory containing the ``*.pro`` file.""" return self.stage.source_path def qmake_args(self): - """Produces a list containing all the arguments that must be passed to - qmake - """ + """List of arguments passed to qmake.""" return [] - def qmake(self, spec, prefix): + def qmake(self, pkg, spec, prefix): """Run ``qmake`` to configure the project and generate a Makefile.""" - with working_dir(self.build_directory): - inspect.getmodule(self).qmake(*self.qmake_args()) + inspect.getmodule(self.pkg).qmake(*self.qmake_args()) - def build(self, spec, prefix): + def build(self, pkg, spec, prefix): """Make the build targets""" - with working_dir(self.build_directory): - inspect.getmodule(self).make() + inspect.getmodule(self.pkg).make() - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Make the install targets""" - with working_dir(self.build_directory): - inspect.getmodule(self).make("install") - - # Tests + inspect.getmodule(self.pkg).make("install") def check(self): - """Searches the Makefile for a ``check:`` target and runs it if found.""" - + """Search the Makefile for a ``check:`` target and runs it if found.""" with working_dir(self.build_directory): self._if_make_target_execute("check") - run_after("build")(PackageBase._run_default_build_time_test_callbacks) - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) + spack.builder.run_after("build")(execute_build_time_tests) diff --git a/lib/spack/spack/build_systems/r.py b/lib/spack/spack/build_systems/r.py index 450cae733b..c62baa3555 100644 --- a/lib/spack/spack/build_systems/r.py +++ b/lib/spack/spack/build_systems/r.py @@ -3,29 +3,63 @@ # # SPDX-License-Identifier: (Apache-2.0 OR MIT) import inspect -from typing import Optional +from typing import Optional, Tuple import llnl.util.lang as lang from spack.directives import extends -from spack.package_base import PackageBase, run_after +from .generic import GenericBuilder, Package -class RPackage(PackageBase): - """Specialized class for packages that are built using R. - - For more information on the R build system, see: - https://stat.ethz.ch/R-manual/R-devel/library/utils/html/INSTALL.html - This class provides a single phase that can be overridden: +class RBuilder(GenericBuilder): + """The R builder provides a single phase that can be overridden: - 1. :py:meth:`~.RPackage.install` + 1. :py:meth:`~.RBuilder.install` It has sensible defaults, and for many packages the only thing - necessary will be to add dependencies + necessary will be to add dependencies. """ - phases = ["install"] + #: Names associated with package methods in the old build-system format + legacy_methods = ( + "configure_args", + "configure_vars", + ) + GenericBuilder.legacy_methods # type: Tuple[str, ...] + + def configure_args(self): + """Arguments to pass to install via ``--configure-args``.""" + return [] + + def configure_vars(self): + """Arguments to pass to install via ``--configure-vars``.""" + return [] + + def install(self, pkg, spec, prefix): + """Installs an R package.""" + + config_args = self.configure_args() + config_vars = self.configure_vars() + + args = ["--vanilla", "CMD", "INSTALL"] + + if config_args: + args.append("--configure-args={0}".format(" ".join(config_args))) + + if config_vars: + args.append("--configure-vars={0}".format(" ".join(config_vars))) + + args.extend(["--library={0}".format(self.pkg.module.r_lib_dir), self.stage.source_path]) + + inspect.getmodule(self.pkg).R(*args) + + +class RPackage(Package): + """Specialized class for packages that are built using R. + + For more information on the R build system, see: + https://stat.ethz.ch/R-manual/R-devel/library/utils/html/INSTALL.html + """ # package attributes that can be expanded to set the homepage, url, # list_url, and git values @@ -35,6 +69,8 @@ class RPackage(PackageBase): # For Bioconductor packages bioc = None # type: Optional[str] + GenericBuilder = RBuilder + maintainers = ["glennpj"] #: This attribute is used in UI queries that need to know the build @@ -70,32 +106,3 @@ class RPackage(PackageBase): def git(self): if self.bioc: return "https://git.bioconductor.org/packages/" + self.bioc - - def configure_args(self): - """Arguments to pass to install via ``--configure-args``.""" - return [] - - def configure_vars(self): - """Arguments to pass to install via ``--configure-vars``.""" - return [] - - def install(self, spec, prefix): - """Installs an R package.""" - - config_args = self.configure_args() - config_vars = self.configure_vars() - - args = ["--vanilla", "CMD", "INSTALL"] - - if config_args: - args.append("--configure-args={0}".format(" ".join(config_args))) - - if config_vars: - args.append("--configure-vars={0}".format(" ".join(config_vars))) - - args.extend(["--library={0}".format(self.module.r_lib_dir), self.stage.source_path]) - - inspect.getmodule(self).R(*args) - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) diff --git a/lib/spack/spack/build_systems/racket.py b/lib/spack/spack/build_systems/racket.py index 7b37d85cf2..889cc07931 100644 --- a/lib/spack/spack/build_systems/racket.py +++ b/lib/spack/spack/build_systems/racket.py @@ -3,14 +3,15 @@ # # SPDX-License-Identifier: (Apache-2.0 OR MIT) import os -from typing import Optional +from typing import Optional, Tuple +import llnl.util.filesystem as fs import llnl.util.lang as lang import llnl.util.tty as tty -from llnl.util.filesystem import working_dir +import spack.builder from spack.build_environment import SPACK_NO_PARALLEL_MAKE, determine_number_of_jobs -from spack.directives import extends +from spack.directives import build_system, extends from spack.package_base import PackageBase from spack.util.environment import env_flag from spack.util.executable import Executable, ProcessError @@ -19,34 +20,52 @@ from spack.util.executable import Executable, ProcessError class RacketPackage(PackageBase): """Specialized class for packages that are built using Racket's `raco pkg install` and `raco setup` commands. - - This class provides the following phases that can be overridden: - - * install - * setup """ #: Package name, version, and extension on PyPI maintainers = ["elfprince13"] - - # Default phases - phases = ["install"] - # To be used in UI queries that require to know which # build-system class we are using build_system_class = "RacketPackage" + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "racket" - extends("racket") + build_system("racket") + + extends("racket", when="build_system=racket") - pkgs = False - subdirectory = None # type: Optional[str] racket_name = None # type: Optional[str] parallel = True @lang.classproperty def homepage(cls): - if cls.pkgs: + if cls.racket_name: return "https://pkgs.racket-lang.org/package/{0}".format(cls.racket_name) + return None + + +@spack.builder.builder("racket") +class RacketBuilder(spack.builder.Builder): + """The Racket builder provides an ``install`` phase that can be overridden.""" + + phases = ("install",) + + #: Names associated with package methods in the old build-system format + legacy_methods = tuple() # type: Tuple[str, ...] + + #: Names associated with package attributes in the old build-system format + legacy_attributes = ("build_directory", "build_time_test_callbacks", "subdirectory") + + #: Callback names for build-time test + build_time_test_callbacks = ["check"] + + racket_name = None # type: Optional[str] + + @property + def subdirectory(self): + if self.racket_name: + return "pkgs/{0}".format(self.pkg.racket_name) + return None @property def build_directory(self): @@ -55,25 +74,25 @@ class RacketPackage(PackageBase): ret = os.path.join(ret, self.subdirectory) return ret - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Install everything from build directory.""" raco = Executable("raco") - with working_dir(self.build_directory): - allow_parallel = self.parallel and (not env_flag(SPACK_NO_PARALLEL_MAKE)) + with fs.working_dir(self.build_directory): + parallel = self.pkg.parallel and (not env_flag(SPACK_NO_PARALLEL_MAKE)) args = [ "pkg", "install", "-t", "dir", "-n", - self.racket_name, + self.pkg.racket_name, "--deps", "fail", "--ignore-implies", "--copy", "-i", "-j", - str(determine_number_of_jobs(allow_parallel)), + str(determine_number_of_jobs(parallel)), "--", os.getcwd(), ] @@ -82,9 +101,8 @@ class RacketPackage(PackageBase): except ProcessError: args.insert(-2, "--skip-installed") raco(*args) - tty.warn( - ( - "Racket package {0} was already installed, uninstalling via " - "Spack may make someone unhappy!" - ).format(self.racket_name) + msg = ( + "Racket package {0} was already installed, uninstalling via " + "Spack may make someone unhappy!" ) + tty.warn(msg.format(self.pkg.racket_name)) diff --git a/lib/spack/spack/build_systems/ruby.py b/lib/spack/spack/build_systems/ruby.py index fcc071f19e..ef29f164ab 100644 --- a/lib/spack/spack/build_systems/ruby.py +++ b/lib/spack/spack/build_systems/ruby.py @@ -2,35 +2,49 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - import glob import inspect -from spack.directives import extends -from spack.package_base import PackageBase, run_after - +import spack.builder +import spack.package_base +from spack.directives import build_system, extends -class RubyPackage(PackageBase): - """Specialized class for building Ruby gems. +from ._checks import BaseBuilder - This class provides two phases that can be overridden if required: - #. :py:meth:`~.RubyPackage.build` - #. :py:meth:`~.RubyPackage.install` - """ +class RubyPackage(spack.package_base.PackageBase): + """Specialized class for building Ruby gems.""" maintainers = ["Kerilk"] - #: Phases of a Ruby package - phases = ["build", "install"] - #: This attribute is used in UI queries that need to know the build #: system base class build_system_class = "RubyPackage" + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "ruby" + + build_system("ruby") + + extends("ruby", when="build_system=ruby") + - extends("ruby") +@spack.builder.builder("ruby") +class RubyBuilder(BaseBuilder): + """The Ruby builder provides two phases that can be overridden if required: - def build(self, spec, prefix): + #. :py:meth:`~.RubyBuilder.build` + #. :py:meth:`~.RubyBuilder.install` + """ + + phases = ("build", "install") + + #: Names associated with package methods in the old build-system format + legacy_methods = () + + #: Names associated with package attributes in the old build-system format + legacy_attributes = () + + def build(self, pkg, spec, prefix): """Build a Ruby gem.""" # ruby-rake provides both rake.gemspec and Rakefile, but only @@ -38,15 +52,15 @@ class RubyPackage(PackageBase): gemspecs = glob.glob("*.gemspec") rakefiles = glob.glob("Rakefile") if gemspecs: - inspect.getmodule(self).gem("build", "--norc", gemspecs[0]) + inspect.getmodule(self.pkg).gem("build", "--norc", gemspecs[0]) elif rakefiles: - jobs = inspect.getmodule(self).make_jobs - inspect.getmodule(self).rake("package", "-j{0}".format(jobs)) + jobs = inspect.getmodule(self.pkg).make_jobs + inspect.getmodule(self.pkg).rake("package", "-j{0}".format(jobs)) else: # Some Ruby packages only ship `*.gem` files, so nothing to build pass - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Install a Ruby gem. The ruby package sets ``GEM_HOME`` to tell gem where to install to.""" @@ -56,9 +70,6 @@ class RubyPackage(PackageBase): # if --install-dir is not used, GEM_PATH is deleted from the # environement, and Gems required to build native extensions will # not be found. Those extensions are built during `gem install`. - inspect.getmodule(self).gem( + inspect.getmodule(self.pkg).gem( "install", "--norc", "--ignore-dependencies", "--install-dir", prefix, gems[0] ) - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) diff --git a/lib/spack/spack/build_systems/scons.py b/lib/spack/spack/build_systems/scons.py index 5470091121..f1ae955917 100644 --- a/lib/spack/spack/build_systems/scons.py +++ b/lib/spack/spack/build_systems/scons.py @@ -2,63 +2,75 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - - import inspect -from spack.directives import depends_on -from spack.package_base import PackageBase, run_after +import spack.builder +import spack.package_base +from spack.directives import build_system, depends_on + +from ._checks import BaseBuilder, execute_build_time_tests -class SConsPackage(PackageBase): +class SConsPackage(spack.package_base.PackageBase): """Specialized class for packages built using SCons. See http://scons.org/documentation.html for more information. - - This class provides the following phases that can be overridden: - - 1. :py:meth:`~.SConsPackage.build` - 2. :py:meth:`~.SConsPackage.install` - - Packages that use SCons as a build system are less uniform than packages - that use other build systems. Developers can add custom subcommands or - variables that control the build. You will likely need to override - :py:meth:`~.SConsPackage.build_args` to pass the appropriate variables. """ - #: Phases of a SCons package - phases = ["build", "install"] - #: To be used in UI queries that require to know which #: build-system class we are using build_system_class = "SConsPackage" #: Callback names for build-time test build_time_test_callbacks = ["build_test"] + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "scons" + + build_system("scons") + + depends_on("scons", type="build", when="build_system=scons") + - depends_on("scons", type="build") +@spack.builder.builder("scons") +class SConsBuilder(BaseBuilder): + """The Scons builder provides the following phases that can be overridden: - def build_args(self, spec, prefix): + 1. :py:meth:`~.SConsBuilder.build` + 2. :py:meth:`~.SConsBuilder.install` + + Packages that use SCons as a build system are less uniform than packages that use + other build systems. Developers can add custom subcommands or variables that + control the build. You will likely need to override + :py:meth:`~.SConsBuilder.build_args` to pass the appropriate variables. + """ + + #: Phases of a SCons package + phases = ("build", "install") + + #: Names associated with package methods in the old build-system format + legacy_methods = ("build_args", "install_args", "build_test") + + #: Names associated with package attributes in the old build-system format + legacy_attributes = () + + def build_args(self): """Arguments to pass to build.""" return [] - def build(self, spec, prefix): + def build(self, pkg, spec, prefix): """Build the package.""" - args = self.build_args(spec, prefix) - - inspect.getmodule(self).scons(*args) + args = self.build_args() + inspect.getmodule(self.pkg).scons(*args) - def install_args(self, spec, prefix): + def install_args(self): """Arguments to pass to install.""" return [] - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Install the package.""" - args = self.install_args(spec, prefix) + args = self.install_args() - inspect.getmodule(self).scons("install", *args) - - # Testing + inspect.getmodule(self.pkg).scons("install", *args) def build_test(self): """Run unit tests after build. @@ -68,7 +80,4 @@ class SConsPackage(PackageBase): """ pass - run_after("build")(PackageBase._run_default_build_time_test_callbacks) - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) + spack.builder.run_after("build")(execute_build_time_tests) diff --git a/lib/spack/spack/build_systems/sip.py b/lib/spack/spack/build_systems/sip.py index 4d16f6731e..b129ca4e0a 100644 --- a/lib/spack/spack/build_systems/sip.py +++ b/lib/spack/spack/build_systems/sip.py @@ -2,7 +2,6 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - import inspect import os import re @@ -10,28 +9,20 @@ import re import llnl.util.tty as tty from llnl.util.filesystem import find, join_path, working_dir -from spack.directives import depends_on, extends -from spack.package_base import PackageBase, run_after +import spack.builder +import spack.package_base +from spack.directives import build_system, depends_on, extends +from spack.multimethod import when + +from ._checks import BaseBuilder, execute_install_time_tests -class SIPPackage(PackageBase): +class SIPPackage(spack.package_base.PackageBase): """Specialized class for packages that are built using the SIP build system. See https://www.riverbankcomputing.com/software/sip/intro for more information. - - This class provides the following phases that can be overridden: - - * configure - * build - * install - - The configure phase already adds a set of default flags. To see more - options, run ``python configure.py --help``. """ - # Default phases - phases = ["configure", "build", "install"] - # To be used in UI queries that require to know which # build-system class we are using build_system_class = "SIPPackage" @@ -41,11 +32,15 @@ class SIPPackage(PackageBase): #: Callback names for install-time test install_time_test_callbacks = ["test"] + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "sip" - extends("python") + build_system("sip") - depends_on("qt") - depends_on("py-sip") + with when("build_system=sip"): + extends("python") + depends_on("qt") + depends_on("py-sip") @property def import_modules(self): @@ -95,11 +90,51 @@ class SIPPackage(PackageBase): """The python ``Executable``.""" inspect.getmodule(self).python(*args, **kwargs) + def test(self): + """Attempts to import modules of the installed package.""" + + # Make sure we are importing the installed modules, + # not the ones in the source directory + for module in self.import_modules: + self.run_test( + inspect.getmodule(self).python.path, + ["-c", "import {0}".format(module)], + purpose="checking import of {0}".format(module), + work_dir="spack-test", + ) + + +@spack.builder.builder("sip") +class SIPBuilder(BaseBuilder): + """The SIP builder provides the following phases that can be overridden: + + * configure + * build + * install + + The configure phase already adds a set of default flags. To see more + options, run ``python configure.py --help``. + """ + + phases = ("configure", "build", "install") + + #: Names associated with package methods in the old build-system format + legacy_methods = ("configure_file", "configure_args", "build_args", "install_args") + + #: Names associated with package attributes in the old build-system format + legacy_attributes = ( + "build_targets", + "install_targets", + "build_time_test_callbacks", + "install_time_test_callbacks", + "build_directory", + ) + def configure_file(self): """Returns the name of the configure file to use.""" return "configure.py" - def configure(self, spec, prefix): + def configure(self, pkg, spec, prefix): """Configure the package.""" configure = self.configure_file() @@ -118,7 +153,7 @@ class SIPPackage(PackageBase): "--bindir", prefix.bin, "--destdir", - inspect.getmodule(self).python_platlib, + inspect.getmodule(self.pkg).python_platlib, ] ) @@ -128,53 +163,35 @@ class SIPPackage(PackageBase): """Arguments to pass to configure.""" return [] - def build(self, spec, prefix): + def build(self, pkg, spec, prefix): """Build the package.""" args = self.build_args() - inspect.getmodule(self).make(*args) + inspect.getmodule(self.pkg).make(*args) def build_args(self): """Arguments to pass to build.""" return [] - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Install the package.""" args = self.install_args() - inspect.getmodule(self).make("install", parallel=False, *args) + inspect.getmodule(self.pkg).make("install", parallel=False, *args) def install_args(self): """Arguments to pass to install.""" return [] - # Testing - - def test(self): - """Attempts to import modules of the installed package.""" - - # Make sure we are importing the installed modules, - # not the ones in the source directory - for module in self.import_modules: - self.run_test( - inspect.getmodule(self).python.path, - ["-c", "import {0}".format(module)], - purpose="checking import of {0}".format(module), - work_dir="spack-test", - ) - - run_after("install")(PackageBase._run_default_install_time_test_callbacks) - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) + spack.builder.run_after("install")(execute_install_time_tests) - @run_after("install") + @spack.builder.run_after("install") def extend_path_setup(self): # See github issue #14121 and PR #15297 - module = self.spec["py-sip"].variants["module"].value + module = self.pkg.spec["py-sip"].variants["module"].value if module != "sip": module = module.split(".")[0] - with working_dir(inspect.getmodule(self).python_platlib): + with working_dir(inspect.getmodule(self.pkg).python_platlib): with open(os.path.join(module, "__init__.py"), "a") as f: f.write("from pkgutil import extend_path\n") f.write("__path__ = extend_path(__path__, __name__)\n") diff --git a/lib/spack/spack/build_systems/waf.py b/lib/spack/spack/build_systems/waf.py index 3571ffd525..f972d4e286 100644 --- a/lib/spack/spack/build_systems/waf.py +++ b/lib/spack/spack/build_systems/waf.py @@ -2,21 +2,38 @@ # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) - - import inspect from llnl.util.filesystem import working_dir -from spack.directives import depends_on -from spack.package_base import PackageBase, run_after +import spack.builder +import spack.package_base +from spack.directives import build_system, depends_on + +from ._checks import BaseBuilder, execute_build_time_tests, execute_install_time_tests -class WafPackage(PackageBase): +class WafPackage(spack.package_base.PackageBase): """Specialized class for packages that are built using the Waf build system. See https://waf.io/book/ for more information. + """ + + # To be used in UI queries that require to know which + # build-system class we are using + build_system_class = "WafPackage" + #: Legacy buildsystem attribute used to deserialize and install old specs + legacy_buildsystem = "waf" + + build_system("waf") + # Much like AutotoolsPackage does not require automake and autoconf + # to build, WafPackage does not require waf to build. It only requires + # python to run the waf build script. + depends_on("python@2.5:", type="build", when="build_system=waf") - This class provides the following phases that can be overridden: + +@spack.builder.builder("waf") +class WafBuilder(BaseBuilder): + """The WAF builder provides the following phases that can be overridden: * configure * build @@ -40,12 +57,25 @@ class WafPackage(PackageBase): function, which passes ``--prefix=/path/to/installation/prefix``. """ - # Default phases - phases = ["configure", "build", "install"] - - # To be used in UI queries that require to know which - # build-system class we are using - build_system_class = "WafPackage" + phases = ("configure", "build", "install") + + #: Names associated with package methods in the old build-system format + legacy_methods = ( + "build_test", + "install_test", + "configure_args", + "build_args", + "install_args", + "build_test", + "install_test", + ) + + #: Names associated with package attributes in the old build-system format + legacy_attributes = ( + "build_time_test_callbacks", + "build_time_test_callbacks", + "build_directory", + ) # Callback names for build-time test build_time_test_callbacks = ["build_test"] @@ -53,11 +83,6 @@ class WafPackage(PackageBase): # Callback names for install-time test install_time_test_callbacks = ["install_test"] - # Much like AutotoolsPackage does not require automake and autoconf - # to build, WafPackage does not require waf to build. It only requires - # python to run the waf build script. - depends_on("python@2.5:", type="build") - @property def build_directory(self): """The directory containing the ``waf`` file.""" @@ -65,18 +90,18 @@ class WafPackage(PackageBase): def python(self, *args, **kwargs): """The python ``Executable``.""" - inspect.getmodule(self).python(*args, **kwargs) + inspect.getmodule(self.pkg).python(*args, **kwargs) def waf(self, *args, **kwargs): """Runs the waf ``Executable``.""" - jobs = inspect.getmodule(self).make_jobs + jobs = inspect.getmodule(self.pkg).make_jobs with working_dir(self.build_directory): self.python("waf", "-j{0}".format(jobs), *args, **kwargs) - def configure(self, spec, prefix): + def configure(self, pkg, spec, prefix): """Configures the project.""" - args = ["--prefix={0}".format(self.prefix)] + args = ["--prefix={0}".format(self.pkg.prefix)] args += self.configure_args() self.waf("configure", *args) @@ -85,7 +110,7 @@ class WafPackage(PackageBase): """Arguments to pass to configure.""" return [] - def build(self, spec, prefix): + def build(self, pkg, spec, prefix): """Executes the build.""" args = self.build_args() @@ -95,7 +120,7 @@ class WafPackage(PackageBase): """Arguments to pass to build.""" return [] - def install(self, spec, prefix): + def install(self, pkg, spec, prefix): """Installs the targets on the system.""" args = self.install_args() @@ -105,8 +130,6 @@ class WafPackage(PackageBase): """Arguments to pass to install.""" return [] - # Testing - def build_test(self): """Run unit tests after build. @@ -115,7 +138,7 @@ class WafPackage(PackageBase): """ pass - run_after("build")(PackageBase._run_default_build_time_test_callbacks) + spack.builder.run_after("build")(execute_build_time_tests) def install_test(self): """Run unit tests after install. @@ -125,7 +148,4 @@ class WafPackage(PackageBase): """ pass - run_after("install")(PackageBase._run_default_install_time_test_callbacks) - - # Check that self.prefix is there after installation - run_after("install")(PackageBase.sanity_check_prefix) + spack.builder.run_after("install")(execute_install_time_tests) diff --git a/lib/spack/spack/builder.py b/lib/spack/spack/builder.py new file mode 100644 index 0000000000..6baa321eea --- /dev/null +++ b/lib/spack/spack/builder.py @@ -0,0 +1,574 @@ +# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other +# Spack Project Developers. See the top-level COPYRIGHT file for details. +# +# SPDX-License-Identifier: (Apache-2.0 OR MIT) +import collections +import copy +import functools +import inspect +from typing import List, Optional, Tuple + +import six + +import llnl.util.compat + +import spack.build_environment + +#: Builder classes, as registered by the "builder" decorator +BUILDER_CLS = {} + +#: An object of this kind is a shared global state used to collect callbacks during +#: class definition time, and is flushed when the class object is created at the end +#: of the class definition +#: +#: Args: +#: attribute_name (str): name of the attribute that will be attached to the builder +#: callbacks (list): container used to temporarily aggregate the callbacks +CallbackTemporaryStage = collections.namedtuple( + "CallbackTemporaryStage", ["attribute_name", "callbacks"] +) + +#: Shared global state to aggregate "@run_before" callbacks +_RUN_BEFORE = CallbackTemporaryStage(attribute_name="run_before_callbacks", callbacks=[]) +#: Shared global state to aggregate "@run_after" callbacks +_RUN_AFTER = CallbackTemporaryStage(attribute_name="run_after_callbacks", callbacks=[]) + +#: Map id(pkg) to a builder, to avoid creating multiple +#: builders for the same package object. +_BUILDERS = {} + + +def builder(build_system_name): + """Class decorator used to register the default builder + for a given build-system. + + Args: + build_system_name (str): name of the build-system + """ + + def _decorator(cls): + cls.build_system = build_system_name + BUILDER_CLS[build_system_name] = cls + return cls + + return _decorator + + +def create(pkg): + """Given a package object with an associated concrete spec, + return the builder object that can install it. + + Args: + pkg (spack.package_base.PackageBase): package for which we want the builder + """ + if id(pkg) not in _BUILDERS: + _BUILDERS[id(pkg)] = _create(pkg) + return _BUILDERS[id(pkg)] + + +class _PhaseAdapter(object): + def __init__(self, builder, phase_fn): + self.builder = builder + self.phase_fn = phase_fn + + def __call__(self, spec, prefix): + return self.phase_fn(self.builder.pkg, spec, prefix) + + +def _create(pkg): + """Return a new builder object for the package object being passed as argument. + + The function inspects the build-system used by the package object and try to: + + 1. Return a custom builder, if any is defined in the same ``package.py`` file. + 2. Return a customization of more generic builders, if any is defined in the + class hierarchy (look at AspellDictPackage for an example of that) + 3. Return a run-time generated adapter builder otherwise + + The run-time generated adapter builder is capable of adapting an old-style package + to the new architecture, where the installation procedure has been extracted from + the ``*Package`` hierarchy into a ``*Builder`` hierarchy. This means that the + adapter looks for attribute or method overrides preferably in the ``*Package`` + before using the default builder implementation. + + Note that in case a builder is explicitly coded in ``package.py``, no attempt is made + to look for build-related methods in the ``*Package``. + + Args: + pkg (spack.package_base.PackageBase): package object for which we need a builder + """ + package_module = inspect.getmodule(pkg) + package_buildsystem = buildsystem_name(pkg) + default_builder_cls = BUILDER_CLS[package_buildsystem] + builder_cls_name = default_builder_cls.__name__ + builder_cls = getattr(package_module, builder_cls_name, None) + if builder_cls: + return builder_cls(pkg) + + # Specialized version of a given buildsystem can subclass some + # base classes and specialize certain phases or methods or attributes. + # In that case they can store their builder class as a class level attribute. + # See e.g. AspellDictPackage as an example. + base_cls = getattr(pkg, builder_cls_name, default_builder_cls) + + # From here on we define classes to construct a special builder that adapts to the + # old, single class, package format. The adapter forwards any call or access to an + # attribute related to the installation procedure to a package object wrapped in + # a class that falls-back on calling the base builder if no override is found on the + # package. The semantic should be the same as the method in the base builder were still + # present in the base class of the package. + + class _ForwardToBaseBuilder(object): + def __init__(self, wrapped_pkg_object, root_builder): + self.wrapped_package_object = wrapped_pkg_object + self.root_builder = root_builder + + package_cls = type(wrapped_pkg_object) + wrapper_cls = type(self) + bases = (package_cls, wrapper_cls) + new_cls_name = package_cls.__name__ + "Wrapper" + new_cls = type(new_cls_name, bases, {}) + new_cls.__module__ = package_cls.__module__ + self.__class__ = new_cls + self.__dict__.update(wrapped_pkg_object.__dict__) + + def __getattr__(self, item): + result = getattr(super(type(self.root_builder), self.root_builder), item) + if item in super(type(self.root_builder), self.root_builder).phases: + result = _PhaseAdapter(self.root_builder, result) + return result + + def forward_method_to_getattr(fn_name): + def __forward(self, *args, **kwargs): + return self.__getattr__(fn_name)(*args, **kwargs) + + return __forward + + # Add fallback methods for the Package object to refer to the builder. If a method + # with the same name is defined in the Package, it will override this definition + # (when _ForwardToBaseBuilder is initialized) + for method_name in ( + base_cls.phases + + base_cls.legacy_methods + + getattr(base_cls, "legacy_long_methods", tuple()) + + ("setup_build_environment", "setup_dependent_build_environment") + ): + setattr(_ForwardToBaseBuilder, method_name, forward_method_to_getattr(method_name)) + + def forward_property_to_getattr(property_name): + def __forward(self): + return self.__getattr__(property_name) + + return __forward + + for attribute_name in base_cls.legacy_attributes: + setattr( + _ForwardToBaseBuilder, + attribute_name, + property(forward_property_to_getattr(attribute_name)), + ) + + class Adapter(six.with_metaclass(_PackageAdapterMeta, base_cls)): + def __init__(self, pkg): + # Deal with custom phases in packages here + if hasattr(pkg, "phases"): + self.phases = pkg.phases + for phase in self.phases: + setattr(Adapter, phase, _PackageAdapterMeta.phase_method_adapter(phase)) + + # Attribute containing the package wrapped in dispatcher with a `__getattr__` + # method that will forward certain calls to the default builder. + self.pkg_with_dispatcher = _ForwardToBaseBuilder(pkg, root_builder=self) + super(Adapter, self).__init__(pkg) + + # These two methods don't follow the (self, spec, prefix) signature of phases nor + # the (self) signature of methods, so they are added explicitly to avoid using a + # catch-all (*args, **kwargs) + def setup_build_environment(self, env): + return self.pkg_with_dispatcher.setup_build_environment(env) + + def setup_dependent_build_environment(self, env, dependent_spec): + return self.pkg_with_dispatcher.setup_dependent_build_environment(env, dependent_spec) + + return Adapter(pkg) + + +def buildsystem_name(pkg): + """Given a package object with an associated concrete spec, + return the name of its build system. + + Args: + pkg (spack.package_base.PackageBase): package for which we want + the build system name + """ + try: + return pkg.spec.variants["build_system"].value + except KeyError: + # We are reading an old spec without the build_system variant + return pkg.legacy_buildsystem + + +class PhaseCallbacksMeta(type): + """Permit to register arbitrary functions during class definition and run them + later, before or after a given install phase. + + Each method decorated with ``run_before`` or ``run_after`` gets temporarily + stored in a global shared state when a class being defined is parsed by the Python + interpreter. At class definition time that temporary storage gets flushed and a list + of callbacks is attached to the class being defined. + """ + + def __new__(mcs, name, bases, attr_dict): + for temporary_stage in (_RUN_BEFORE, _RUN_AFTER): + staged_callbacks = temporary_stage.callbacks + + # We don't have callbacks in this class, move on + if not staged_callbacks: + continue + + # If we are here we have callbacks. To get a complete list, get first what + # was attached to parent classes, then prepend what we have registered here. + # + # The order should be: + # 1. Callbacks are registered in order within the same class + # 2. Callbacks defined in derived classes precede those defined in base + # classes + for base in bases: + callbacks_from_base = getattr(base, temporary_stage.attribute_name, None) + if callbacks_from_base: + break + callbacks_from_base = callbacks_from_base or [] + + # Set the callbacks in this class and flush the temporary stage + attr_dict[temporary_stage.attribute_name] = staged_callbacks[:] + callbacks_from_base + del temporary_stage.callbacks[:] + + return super(PhaseCallbacksMeta, mcs).__new__(mcs, name, bases, attr_dict) + + @staticmethod + def run_after(phase, when=None): + """Decorator to register a function for running after a given phase. + + Args: + phase (str): phase after which the function must run. + when (str): condition under which the function is run (if None, it is always run). + """ + + def _decorator(fn): + key = (phase, when) + item = (key, fn) + _RUN_AFTER.callbacks.append(item) + return fn + + return _decorator + + @staticmethod + def run_before(phase, when=None): + """Decorator to register a function for running before a given phase. + + Args: + phase (str): phase before which the function must run. + when (str): condition under which the function is run (if None, it is always run). + """ + + def _decorator(fn): + key = (phase, when) + item = (key, fn) + _RUN_BEFORE.callbacks.append(item) + return fn + + return _decorator + + +class BuilderMeta(PhaseCallbacksMeta, type(llnl.util.compat.Sequence)): # type: ignore + pass + + +class _PackageAdapterMeta(BuilderMeta): + """Metaclass to adapt old-style packages to the new architecture based on builders + for the installation phase. + + This class does the necessary mangling to function argument so that a call to a + builder object can delegate to a package object. + """ + + @staticmethod + def phase_method_adapter(phase_name): + def _adapter(self, pkg, spec, prefix): + phase_fn = getattr(self.pkg_with_dispatcher, phase_name) + return phase_fn(spec, prefix) + + return _adapter + + @staticmethod + def legacy_long_method_adapter(method_name): + def _adapter(self, spec, prefix): + bind_method = getattr(self.pkg_with_dispatcher, method_name) + return bind_method(spec, prefix) + + return _adapter + + @staticmethod + def legacy_method_adapter(method_name): + def _adapter(self): + bind_method = getattr(self.pkg_with_dispatcher, method_name) + return bind_method() + + return _adapter + + @staticmethod + def legacy_attribute_adapter(attribute_name): + def _adapter(self): + return getattr(self.pkg_with_dispatcher, attribute_name) + + return property(_adapter) + + @staticmethod + def combine_callbacks(pipeline_attribute_name): + """This function combines callbacks from old-style packages with callbacks that might + be registered for the default builder. + + It works by: + 1. Extracting the callbacks from the old-style package + 2. Transforming those callbacks by adding an adapter that receives a builder as argument + and calls the wrapped function with ``builder.pkg`` + 3. Combining the list of transformed callbacks with those that might be present in the + default builder + """ + + def _adapter(self): + def unwrap_pkg(fn): + @functools.wraps(fn) + def _wrapped(builder): + return fn(builder.pkg_with_dispatcher) + + return _wrapped + + # Concatenate the current list with the one from package + callbacks_from_package = getattr(self.pkg, pipeline_attribute_name, []) + callbacks_from_package = [(key, unwrap_pkg(x)) for key, x in callbacks_from_package] + callbacks_from_builder = getattr(super(type(self), self), pipeline_attribute_name, []) + return callbacks_from_package + callbacks_from_builder + + return property(_adapter) + + def __new__(mcs, name, bases, attr_dict): + # Add ways to intercept methods and attribute calls and dispatch + # them first to a package object + default_builder_cls = bases[0] + for phase_name in default_builder_cls.phases: + attr_dict[phase_name] = _PackageAdapterMeta.phase_method_adapter(phase_name) + + for method_name in default_builder_cls.legacy_methods: + attr_dict[method_name] = _PackageAdapterMeta.legacy_method_adapter(method_name) + + # These exist e.g. for Python, see discussion in https://github.com/spack/spack/pull/32068 + for method_name in getattr(default_builder_cls, "legacy_long_methods", []): + attr_dict[method_name] = _PackageAdapterMeta.legacy_long_method_adapter(method_name) + + for attribute_name in default_builder_cls.legacy_attributes: + attr_dict[attribute_name] = _PackageAdapterMeta.legacy_attribute_adapter( + attribute_name + ) + + combine_callbacks = _PackageAdapterMeta.combine_callbacks + attr_dict[_RUN_BEFORE.attribute_name] = combine_callbacks(_RUN_BEFORE.attribute_name) + attr_dict[_RUN_AFTER.attribute_name] = combine_callbacks(_RUN_AFTER.attribute_name) + + return super(_PackageAdapterMeta, mcs).__new__(mcs, name, bases, attr_dict) + + +class InstallationPhase(object): + """Manages a single phase of the installation. + + This descriptor stores at creation time the name of the method it should + search for execution. The method is retrieved at __get__ time, so that + it can be overridden by subclasses of whatever class declared the phases. + + It also provides hooks to execute arbitrary callbacks before and after + the phase. + """ + + def __init__(self, name, builder): + self.name = name + self.builder = builder + self.phase_fn = self._select_phase_fn() + self.run_before = self._make_callbacks(_RUN_BEFORE.attribute_name) + self.run_after = self._make_callbacks(_RUN_AFTER.attribute_name) + + def _make_callbacks(self, callbacks_attribute): + result = [] + callbacks = getattr(self.builder, callbacks_attribute, []) + for (phase, condition), fn in callbacks: + # Same if it is for another phase + if phase != self.name: + continue + + # If we have no condition or the callback satisfies a condition, register it + if condition is None or self.builder.pkg.spec.satisfies(condition): + result.append(fn) + return result + + def __str__(self): + msg = '{0}: executing "{1}" phase' + return msg.format(self.builder, self.name) + + def execute(self): + pkg = self.builder.pkg + self._on_phase_start(pkg) + + for callback in self.run_before: + callback(self.builder) + + self.phase_fn(pkg, pkg.spec, pkg.prefix) + + for callback in self.run_after: + callback(self.builder) + + self._on_phase_exit(pkg) + + def _select_phase_fn(self): + phase_fn = getattr(self.builder, self.name, None) + + if not phase_fn: + msg = ( + 'unexpected error: package "{0.fullname}" must implement an ' + '"{1}" phase for the "{2}" build system' + ) + raise RuntimeError(msg.format(self.builder.pkg, self.name, self.builder.build_system)) + + return phase_fn + + def _on_phase_start(self, instance): + # If a phase has a matching stop_before_phase attribute, + # stop the installation process raising a StopPhase + if getattr(instance, "stop_before_phase", None) == self.name: + raise spack.build_environment.StopPhase( + "Stopping before '{0}' phase".format(self.name) + ) + + def _on_phase_exit(self, instance): + # If a phase has a matching last_phase attribute, + # stop the installation process raising a StopPhase + if getattr(instance, "last_phase", None) == self.name: + raise spack.build_environment.StopPhase("Stopping at '{0}' phase".format(self.name)) + + def copy(self): + return copy.deepcopy(self) + + +class Builder(six.with_metaclass(BuilderMeta, llnl.util.compat.Sequence)): + """A builder is a class that, given a package object (i.e. associated with + concrete spec), knows how to install it. + + The builder behaves like a sequence, and when iterated over return the + "phases" of the installation in the correct order. + + Args: + pkg (spack.package_base.PackageBase): package object to be built + """ + + #: Sequence of phases. Must be defined in derived classes + phases = None # type: Optional[Tuple[str, ...]] + #: Build system name. Must also be defined in derived classes. + build_system = None # type: Optional[str] + + legacy_methods = () # type: Tuple[str, ...] + legacy_attributes = () # type: Tuple[str, ...] + + #: List of glob expressions. Each expression must either be + #: absolute or relative to the package source path. + #: Matching artifacts found at the end of the build process will be + #: copied in the same directory tree as _spack_build_logfile and + #: _spack_build_envfile. + archive_files = [] # type: List[str] + + def __init__(self, pkg): + self.pkg = pkg + self.callbacks = {} + for phase in self.phases: + self.callbacks[phase] = InstallationPhase(phase, self) + + @property + def spec(self): + return self.pkg.spec + + @property + def stage(self): + return self.pkg.stage + + @property + def prefix(self): + return self.pkg.prefix + + def test(self): + # Defer tests to virtual and concrete packages + pass + + def setup_build_environment(self, env): + """Sets up the build environment for a package. + + This method will be called before the current package prefix exists in + Spack's store. + + Args: + env (spack.util.environment.EnvironmentModifications): environment + modifications to be applied when the package is built. Package authors + can call methods on it to alter the build environment. + """ + if not hasattr(super(Builder, self), "setup_build_environment"): + return + super(Builder, self).setup_build_environment(env) + + def setup_dependent_build_environment(self, env, dependent_spec): + """Sets up the build environment of packages that depend on this one. + + This is similar to ``setup_build_environment``, but it is used to + modify the build environments of packages that *depend* on this one. + + This gives packages like Python and others that follow the extension + model a way to implement common environment or compile-time settings + for dependencies. + + This method will be called before the dependent package prefix exists + in Spack's store. + + Examples: + 1. Installing python modules generally requires ``PYTHONPATH`` + to point to the ``lib/pythonX.Y/site-packages`` directory in the + module's install prefix. This method could be used to set that + variable. + + Args: + env (spack.util.environment.EnvironmentModifications): environment + modifications to be applied when the dependent package is built. + Package authors can call methods on it to alter the build environment. + + dependent_spec (spack.spec.Spec): the spec of the dependent package + about to be built. This allows the extendee (self) to query + the dependent's state. Note that *this* package's spec is + available as ``self.spec`` + """ + if not hasattr(super(Builder, self), "setup_dependent_build_environment"): + return + super(Builder, self).setup_dependent_build_environment(env, dependent_spec) + + def __getitem__(self, idx): + key = self.phases[idx] + return self.callbacks[key] + + def __len__(self): + return len(self.phases) + + def __repr__(self): + msg = "{0}({1})" + return msg.format(type(self).__name__, self.pkg.spec.format("{name}/{hash:7}")) + + def __str__(self): + msg = '"{0}" builder for "{1}"' + return msg.format(type(self).build_system, self.pkg.spec.format("{name}/{hash:7}")) + + +# Export these names as standalone to be used in packages +run_after = PhaseCallbacksMeta.run_after +run_before = PhaseCallbacksMeta.run_before diff --git a/lib/spack/spack/cmd/info.py b/lib/spack/spack/cmd/info.py index ce69d98dbb..6a16a153ea 100644 --- a/lib/spack/spack/cmd/info.py +++ b/lib/spack/spack/cmd/info.py @@ -210,11 +210,11 @@ def print_maintainers(pkg): def print_phases(pkg): """output installation phases""" - if hasattr(pkg, "phases") and pkg.phases: + if hasattr(pkg.builder, "phases") and pkg.builder.phases: color.cprint("") color.cprint(section_title("Installation Phases:")) phase_str = "" - for phase in pkg.phases: + for phase in pkg.builder.phases: phase_str += " {0}".format(phase) color.cprint(phase_str) diff --git a/lib/spack/spack/cmd/uninstall.py b/lib/spack/spack/cmd/uninstall.py index be4b74c54a..af85b64188 100644 --- a/lib/spack/spack/cmd/uninstall.py +++ b/lib/spack/spack/cmd/uninstall.py @@ -228,7 +228,7 @@ def do_uninstall(env, specs, force): except spack.repo.UnknownEntityError: # The package.py file has gone away -- but still # want to uninstall. - spack.package_base.Package.uninstall_by_spec(item, force=True) + spack.package_base.PackageBase.uninstall_by_spec(item, force=True) # A package is ready to be uninstalled when nothing else references it, # unless we are requested to force uninstall it. diff --git a/lib/spack/spack/detection/common.py b/lib/spack/spack/detection/common.py index d79b62b872..b9cd871fc2 100644 --- a/lib/spack/spack/detection/common.py +++ b/lib/spack/spack/detection/common.py @@ -228,7 +228,7 @@ def compute_windows_program_path_for_package(pkg): program files location, return list of best guesses Args: - pkg (spack.package_base.Package): package for which + pkg (spack.package_base.PackageBase): package for which Program Files location is to be computed """ if not is_windows: diff --git a/lib/spack/spack/directives.py b/lib/spack/spack/directives.py index c7491861eb..7c49158c79 100644 --- a/lib/spack/spack/directives.py +++ b/lib/spack/spack/directives.py @@ -17,6 +17,7 @@ definition to modify the package, for example: The available directives are: + * ``build_system`` * ``conflicts`` * ``depends_on`` * ``extends`` @@ -59,13 +60,15 @@ __all__ = [ "patch", "variant", "resource", + "build_system", ] #: These are variant names used by Spack internally; packages can't use them reserved_names = ["patches", "dev_path"] -#: Names of possible directives. This list is populated elsewhere in the file. -directive_names = [] +#: Names of possible directives. This list is mostly populated using the @directive decorator. +#: Some directives leverage others and in that case are not automatically added. +directive_names = ["build_system"] _patch_order_index = 0 @@ -758,6 +761,17 @@ def resource(**kwargs): return _execute_resource +def build_system(*values, **kwargs): + default = kwargs.get("default", None) or values[0] + return variant( + "build_system", + values=tuple(values), + description="Build systems supported by the package", + default=default, + multi=False, + ) + + class DirectiveError(spack.error.SpackError): """This is raised when something is wrong with a package directive.""" diff --git a/lib/spack/spack/installer.py b/lib/spack/spack/installer.py index 411c076aa8..620f489c77 100644 --- a/lib/spack/spack/installer.py +++ b/lib/spack/spack/installer.py @@ -112,14 +112,15 @@ def _check_last_phase(pkg): Raises: ``BadInstallPhase`` if stop_before or last phase is invalid """ - if pkg.stop_before_phase and pkg.stop_before_phase not in pkg.phases: + phases = pkg.builder.phases + if pkg.stop_before_phase and pkg.stop_before_phase not in phases: raise BadInstallPhase(pkg.name, pkg.stop_before_phase) - if pkg.last_phase and pkg.last_phase not in pkg.phases: + if pkg.last_phase and pkg.last_phase not in phases: raise BadInstallPhase(pkg.name, pkg.last_phase) # If we got a last_phase, make sure it's not already last - if pkg.last_phase and pkg.last_phase == pkg.phases[-1]: + if pkg.last_phase and pkg.last_phase == phases[-1]: pkg.last_phase = None @@ -129,7 +130,7 @@ def _handle_external_and_upstream(pkg, explicit): database if it is external package. Args: - pkg (spack.package_base.Package): the package whose installation is under + pkg (spack.package_base.PackageBase): the package whose installation is under consideration explicit (bool): the package was explicitly requested by the user Return: @@ -559,7 +560,7 @@ def log(pkg): Copy provenance into the install directory on success Args: - pkg (spack.package_base.Package): the package that was built and installed + pkg (spack.package_base.PackageBase): the package that was built and installed """ packages_dir = spack.store.layout.build_packages_path(pkg.spec) @@ -596,7 +597,7 @@ def log(pkg): errors = six.StringIO() target_dir = os.path.join(spack.store.layout.metadata_path(pkg.spec), "archived-files") - for glob_expr in pkg.archive_files: + for glob_expr in pkg.builder.archive_files: # Check that we are trying to copy things that are # in the stage tree (not arbitrary files) abs_expr = os.path.realpath(glob_expr) @@ -810,7 +811,7 @@ class PackageInstaller(object): Creates and queus the initial build task for the package. Args: - pkg (spack.package_base.Package): the package to be built and installed + pkg (spack.package_base.PackageBase): the package to be built and installed request (BuildRequest or None): the associated install request where ``None`` can be used to indicate the package was explicitly requested by the user @@ -1404,7 +1405,7 @@ class PackageInstaller(object): Write a small metadata file with the current spack environment. Args: - pkg (spack.package_base.Package): the package to be built and installed + pkg (spack.package_base.PackageBase): the package to be built and installed """ if not os.path.exists(pkg.spec.prefix): path = spack.util.path.debug_padded_filter(pkg.spec.prefix) @@ -1477,8 +1478,8 @@ class PackageInstaller(object): known dependents. Args: - pkg (spack.package_base.Package): Package that has been installed locally, - externally or upstream + pkg (spack.package_base.PackageBase): Package that has been installed + locally, externally or upstream dependent_ids (list or None): list of the package's dependent ids, or None if the dependent ids are limited to those maintained in the package (dependency DAG) @@ -1562,11 +1563,7 @@ class PackageInstaller(object): return InstallAction.OVERWRITE def install(self): - """ - Install the requested package(s) and or associated dependencies. - - Args: - pkg (spack.package_base.Package): the package to be built and installed""" + """Install the requested package(s) and or associated dependencies.""" self._init_queue() fail_fast_err = "Terminating after first install failure" @@ -1951,6 +1948,8 @@ class BuildProcessInstaller(object): fs.install_tree(pkg.stage.source_path, src_target) def _real_install(self): + import spack.builder + pkg = self.pkg # Do the real install in the source directory. @@ -1981,13 +1980,11 @@ class BuildProcessInstaller(object): # Spawn a daemon that reads from a pipe and redirects # everything to log_path, and provide the phase for logging - for i, (phase_name, phase_attr) in enumerate( - zip(pkg.phases, pkg._InstallPhase_phases) - ): - + builder = spack.builder.create(pkg) + for i, phase_fn in enumerate(builder): # Keep a log file for each phase log_dir = os.path.dirname(pkg.log_path) - log_file = "spack-build-%02d-%s-out.txt" % (i + 1, phase_name.lower()) + log_file = "spack-build-%02d-%s-out.txt" % (i + 1, phase_fn.name.lower()) log_file = os.path.join(log_dir, log_file) try: @@ -2005,20 +2002,20 @@ class BuildProcessInstaller(object): with logger.force_echo(): inner_debug_level = tty.debug_level() tty.set_debug(debug_level) - tty.msg("{0} Executing phase: '{1}'".format(self.pre, phase_name)) + msg = "{0} Executing phase: '{1}'" + tty.msg(msg.format(self.pre, phase_fn.name)) tty.set_debug(inner_debug_level) # Redirect stdout and stderr to daemon pipe - phase = getattr(pkg, phase_attr) - self.timer.phase(phase_name) + self.timer.phase(phase_fn.name) # Catch any errors to report to logging - phase(pkg.spec, pkg.prefix) - spack.hooks.on_phase_success(pkg, phase_name, log_file) + phase_fn.execute() + spack.hooks.on_phase_success(pkg, phase_fn.name, log_file) except BaseException: combine_phase_logs(pkg.phase_log_files, pkg.log_path) - spack.hooks.on_phase_error(pkg, phase_name, log_file) + spack.hooks.on_phase_error(pkg, phase_fn.name, log_file) # phase error indicates install error spack.hooks.on_install_failure(pkg.spec) @@ -2094,7 +2091,7 @@ class BuildTask(object): Instantiate a build task for a package. Args: - pkg (spack.package_base.Package): the package to be built and installed + pkg (spack.package_base.PackageBase): the package to be built and installed request (BuildRequest or None): the associated install request where ``None`` can be used to indicate the package was explicitly requested by the user @@ -2310,7 +2307,7 @@ class BuildRequest(object): Instantiate a build request for a package. Args: - pkg (spack.package_base.Package): the package to be built and installed + pkg (spack.package_base.PackageBase): the package to be built and installed install_args (dict): the install arguments associated with ``pkg`` """ # Ensure dealing with a package that has a concrete spec diff --git a/lib/spack/spack/mixins.py b/lib/spack/spack/mixins.py index ace3681e52..b43b85aa02 100644 --- a/lib/spack/spack/mixins.py +++ b/lib/spack/spack/mixins.py @@ -6,10 +6,9 @@ """This module contains additional behavior that can be attached to any given package. """ -import collections import os import sys -from typing import Callable, DefaultDict, Dict, List # novm +from typing import Callable, DefaultDict, List # novm if sys.version_info >= (3, 5): CallbackDict = DefaultDict[str, List[Callable]] @@ -18,105 +17,7 @@ else: import llnl.util.filesystem -__all__ = [ - "filter_compiler_wrappers", - "PackageMixinsMeta", -] - - -class PackageMixinsMeta(type): - """This metaclass serves the purpose of implementing a declarative syntax - for package mixins. - - Mixins are implemented below in the form of a function. Each one of them - needs to register a callable that takes a single argument to be run - before or after a certain phase. This callable is basically a method that - gets implicitly attached to the package class by calling the mixin. - """ - - _methods_to_be_added = {} # type: Dict[str, Callable] - _add_method_before = collections.defaultdict(list) # type: CallbackDict - _add_method_after = collections.defaultdict(list) # type: CallbackDict - - @staticmethod - def register_method_before(fn, phase): # type: (Callable, str) -> None - """Registers a method to be run before a certain phase. - - Args: - fn: function taking a single argument (self) - phase (str): phase before which fn must run - """ - PackageMixinsMeta._methods_to_be_added[fn.__name__] = fn - PackageMixinsMeta._add_method_before[phase].append(fn) - - @staticmethod - def register_method_after(fn, phase): # type: (Callable, str) -> None - """Registers a method to be run after a certain phase. - - Args: - fn: function taking a single argument (self) - phase (str): phase after which fn must run - """ - PackageMixinsMeta._methods_to_be_added[fn.__name__] = fn - PackageMixinsMeta._add_method_after[phase].append(fn) - - def __init__(cls, name, bases, attr_dict): - - # Add the methods to the class being created - if PackageMixinsMeta._methods_to_be_added: - attr_dict.update(PackageMixinsMeta._methods_to_be_added) - PackageMixinsMeta._methods_to_be_added.clear() - - attr_fmt = "_InstallPhase_{0}" - - # Copy the phases that needs it to the most derived classes - # in order not to interfere with other packages in the hierarchy - phases_to_be_copied = list(PackageMixinsMeta._add_method_before.keys()) - phases_to_be_copied += list(PackageMixinsMeta._add_method_after.keys()) - - for phase in phases_to_be_copied: - - attr_name = attr_fmt.format(phase) - - # Here we want to get the attribute directly from the class (not - # from the instance), so that we can modify it and add the mixin - # method to the pipeline. - phase = getattr(cls, attr_name) - - # Due to MRO, we may have taken a method from a parent class - # and modifying it may influence other packages in unwanted - # manners. Solve the problem by copying the phase into the most - # derived class. - setattr(cls, attr_name, phase.copy()) - - # Insert the methods in the appropriate position - # in the installation pipeline. - - for phase in PackageMixinsMeta._add_method_before: - - attr_name = attr_fmt.format(phase) - phase_obj = getattr(cls, attr_name) - fn_list = PackageMixinsMeta._add_method_after[phase] - - for f in fn_list: - phase_obj.run_before.append(f) - - # Flush the dictionary for the next class - PackageMixinsMeta._add_method_before.clear() - - for phase in PackageMixinsMeta._add_method_after: - - attr_name = attr_fmt.format(phase) - phase_obj = getattr(cls, attr_name) - fn_list = PackageMixinsMeta._add_method_after[phase] - - for f in fn_list: - phase_obj.run_after.append(f) - - # Flush the dictionary for the next class - PackageMixinsMeta._add_method_after.clear() - - super(PackageMixinsMeta, cls).__init__(name, bases, attr_dict) +import spack.builder def filter_compiler_wrappers(*files, **kwargs): @@ -216,4 +117,4 @@ def filter_compiler_wrappers(*files, **kwargs): if self.compiler.name == "nag": x.filter("-Wl,--enable-new-dtags", "", **filter_kwargs) - PackageMixinsMeta.register_method_after(_filter_compiler_wrappers_impl, after) + spack.builder.run_after(after)(_filter_compiler_wrappers_impl) diff --git a/lib/spack/spack/multimethod.py b/lib/spack/spack/multimethod.py index 2c6a56db1a..66f7a887f3 100644 --- a/lib/spack/spack/multimethod.py +++ b/lib/spack/spack/multimethod.py @@ -120,32 +120,36 @@ class SpecMultiMethod(object): return method return self.default or None - def __call__(self, package_self, *args, **kwargs): + def __call__(self, package_or_builder_self, *args, **kwargs): """Find the first method with a spec that matches the package's spec. If none is found, call the default or if there is none, then raise a NoSuchMethodError. """ - spec_method = self._get_method_by_spec(package_self.spec) + spec_method = self._get_method_by_spec(package_or_builder_self.spec) if spec_method: - return spec_method(package_self, *args, **kwargs) + return spec_method(package_or_builder_self, *args, **kwargs) # Unwrap the MRO of `package_self by hand. Note that we can't # use `super()` here, because using `super()` recursively # requires us to know the class of `package_self`, as well as # its superclasses for successive calls. We don't have that # information within `SpecMultiMethod`, because it is not # associated with the package class. - for cls in inspect.getmro(package_self.__class__)[1:]: + for cls in inspect.getmro(package_or_builder_self.__class__)[1:]: superself = cls.__dict__.get(self.__name__, None) + if isinstance(superself, SpecMultiMethod): # Check parent multimethod for method for spec. - superself_method = superself._get_method_by_spec(package_self.spec) + superself_method = superself._get_method_by_spec(package_or_builder_self.spec) if superself_method: - return superself_method(package_self, *args, **kwargs) + return superself_method(package_or_builder_self, *args, **kwargs) elif superself: - return superself(package_self, *args, **kwargs) + return superself(package_or_builder_self, *args, **kwargs) raise NoSuchMethodError( - type(package_self), self.__name__, package_self.spec, [m[0] for m in self.method_list] + type(package_or_builder_self), + self.__name__, + package_or_builder_self.spec, + [m[0] for m in self.method_list], ) diff --git a/lib/spack/spack/package.py b/lib/spack/spack/package.py index 65e699bda4..46c9da4844 100644 --- a/lib/spack/spack/package.py +++ b/lib/spack/spack/package.py @@ -17,6 +17,7 @@ from llnl.util.filesystem import * import spack.util.executable from spack.build_systems.aspell_dict import AspellDictPackage from spack.build_systems.autotools import AutotoolsPackage +from spack.build_systems.bundle import BundlePackage from spack.build_systems.cached_cmake import ( CachedCMakePackage, cmake_cache_option, @@ -25,12 +26,14 @@ from spack.build_systems.cached_cmake import ( ) from spack.build_systems.cmake import CMakePackage from spack.build_systems.cuda import CudaPackage +from spack.build_systems.generic import Package from spack.build_systems.gnu import GNUMirrorPackage from spack.build_systems.intel import IntelPackage from spack.build_systems.lua import LuaPackage from spack.build_systems.makefile import MakefilePackage from spack.build_systems.maven import MavenPackage from spack.build_systems.meson import MesonPackage +from spack.build_systems.nmake import NMakePackage from spack.build_systems.octave import OctavePackage from spack.build_systems.oneapi import ( IntelOneApiLibraryPackage, @@ -38,7 +41,7 @@ from spack.build_systems.oneapi import ( IntelOneApiStaticLibraryList, ) from spack.build_systems.perl import PerlPackage -from spack.build_systems.python import PythonPackage +from spack.build_systems.python import PythonExtension, PythonPackage from spack.build_systems.qmake import QMakePackage from spack.build_systems.r import RPackage from spack.build_systems.racket import RacketPackage @@ -50,6 +53,7 @@ from spack.build_systems.sourceforge import SourceforgePackage from spack.build_systems.sourceware import SourcewarePackage from spack.build_systems.waf import WafPackage from spack.build_systems.xorg import XorgPackage +from spack.builder import run_after, run_before from spack.dependency import all_deptypes from spack.directives import * from spack.install_test import get_escaped_text_output @@ -62,17 +66,13 @@ from spack.installer import ( from spack.mixins import filter_compiler_wrappers from spack.multimethod import when from spack.package_base import ( - BundlePackage, DependencyConflictError, - Package, build_system_flags, env_flags, flatten_dependencies, inject_flags, install_dependency_symlinks, on_package_attributes, - run_after, - run_before, ) from spack.spec import InvalidSpecDetected, Spec from spack.util.executable import * diff --git a/lib/spack/spack/package_base.py b/lib/spack/spack/package_base.py index f05c0ff868..3c3bcaa84d 100644 --- a/lib/spack/spack/package_base.py +++ b/lib/spack/spack/package_base.py @@ -33,7 +33,7 @@ import six import llnl.util.filesystem as fsys import llnl.util.tty as tty -from llnl.util.lang import classproperty, match_predicate, memoized, nullcontext +from llnl.util.lang import classproperty, memoized, nullcontext from llnl.util.link_tree import LinkTree import spack.compilers @@ -104,7 +104,7 @@ def deprecated_version(pkg, version): """Return True if the version is deprecated, False otherwise. Arguments: - pkg (Package): The package whose version is to be checked. + pkg (PackageBase): The package whose version is to be checked. version (str or spack.version.VersionBase): The version being checked """ if not isinstance(version, VersionBase): @@ -122,7 +122,7 @@ def preferred_version(pkg): Returns a sorted list of the preferred versions of the package. Arguments: - pkg (Package): The package whose versions are to be assessed. + pkg (PackageBase): The package whose versions are to be assessed. """ # Here we sort first on the fact that a version is marked # as preferred in the package, then on the fact that the @@ -131,77 +131,6 @@ def preferred_version(pkg): return sorted(pkg.versions, key=key_fn).pop() -class InstallPhase(object): - """Manages a single phase of the installation. - - This descriptor stores at creation time the name of the method it should - search for execution. The method is retrieved at __get__ time, so that - it can be overridden by subclasses of whatever class declared the phases. - - It also provides hooks to execute arbitrary callbacks before and after - the phase. - """ - - def __init__(self, name): - self.name = name - self.run_before = [] - self.run_after = [] - - def __get__(self, instance, owner): - # The caller is a class that is trying to customize - # my behavior adding something - if instance is None: - return self - # If instance is there the caller wants to execute the - # install phase, thus return a properly set wrapper - phase = getattr(instance, self.name) - - @functools.wraps(phase) - def phase_wrapper(spec, prefix): - # Check instance attributes at the beginning of a phase - self._on_phase_start(instance) - # Execute phase pre-conditions, - # and give them the chance to fail - for callback in self.run_before: - callback(instance) - phase(spec, prefix) - # Execute phase sanity_checks, - # and give them the chance to fail - for callback in self.run_after: - callback(instance) - # Check instance attributes at the end of a phase - self._on_phase_exit(instance) - - return phase_wrapper - - def _on_phase_start(self, instance): - # If a phase has a matching stop_before_phase attribute, - # stop the installation process raising a StopPhase - if getattr(instance, "stop_before_phase", None) == self.name: - from spack.build_environment import StopPhase - - raise StopPhase("Stopping before '{0}' phase".format(self.name)) - - def _on_phase_exit(self, instance): - # If a phase has a matching last_phase attribute, - # stop the installation process raising a StopPhase - if getattr(instance, "last_phase", None) == self.name: - from spack.build_environment import StopPhase - - raise StopPhase("Stopping at '{0}' phase".format(self.name)) - - def copy(self): - try: - return copy.deepcopy(self) - except TypeError: - # This bug-fix was not back-ported in Python 2.6 - # http://bugs.python.org/issue1515 - other = InstallPhase(self.name) - other.run_before.extend(self.run_before) - other.run_after.extend(self.run_after) - return other - - class WindowsRPathMeta(object): """Collection of functionality surrounding Windows RPATH specific features @@ -368,23 +297,18 @@ class DetectablePackageMeta(object): class PackageMeta( + spack.builder.PhaseCallbacksMeta, DetectablePackageMeta, spack.directives.DirectiveMeta, - spack.mixins.PackageMixinsMeta, spack.multimethod.MultiMethodMeta, ): """ Package metaclass for supporting directives (e.g., depends_on) and phases """ - phase_fmt = "_InstallPhase_{0}" - - # These are accessed only through getattr, by name - _InstallPhase_run_before = {} # type: Dict[str, List[Callable]] - _InstallPhase_run_after = {} # type: Dict[str, List[Callable]] - def __new__(cls, name, bases, attr_dict): """ + FIXME: REWRITE Instance creation is preceded by phase attribute transformations. Conveniently transforms attributes to permit extensible phases by @@ -392,70 +316,10 @@ class PackageMeta( InstallPhase attributes in the class that will be initialized in __init__. """ - if "phases" in attr_dict: - # Turn the strings in 'phases' into InstallPhase instances - # and add them as private attributes - _InstallPhase_phases = [PackageMeta.phase_fmt.format(x) for x in attr_dict["phases"]] - for phase_name, callback_name in zip(_InstallPhase_phases, attr_dict["phases"]): - attr_dict[phase_name] = InstallPhase(callback_name) - attr_dict["_InstallPhase_phases"] = _InstallPhase_phases - - def _flush_callbacks(check_name): - # Name of the attribute I am going to check it exists - check_attr = PackageMeta.phase_fmt.format(check_name) - checks = getattr(cls, check_attr) - if checks: - for phase_name, funcs in checks.items(): - phase_attr = PackageMeta.phase_fmt.format(phase_name) - try: - # Search for the phase in the attribute dictionary - phase = attr_dict[phase_attr] - except KeyError: - # If it is not there it's in the bases - # and we added a check. We need to copy - # and extend - for base in bases: - phase = getattr(base, phase_attr, None) - if phase is not None: - break - - phase = attr_dict[phase_attr] = phase.copy() - getattr(phase, check_name).extend(funcs) - # Clear the attribute for the next class - setattr(cls, check_attr, {}) - - _flush_callbacks("run_before") - _flush_callbacks("run_after") - - # Reset names for packages that inherit from another - # package with a different name attr_dict["_name"] = None return super(PackageMeta, cls).__new__(cls, name, bases, attr_dict) - @staticmethod - def register_callback(check_type, *phases): - def _decorator(func): - attr_name = PackageMeta.phase_fmt.format(check_type) - check_list = getattr(PackageMeta, attr_name) - for item in phases: - checks = check_list.setdefault(item, []) - checks.append(func) - setattr(PackageMeta, attr_name, check_list) - return func - - return _decorator - - -def run_before(*phases): - """Registers a method of a package to be run before a given phase""" - return PackageMeta.register_callback("run_before", *phases) - - -def run_after(*phases): - """Registers a method of a package to be run after a given phase""" - return PackageMeta.register_callback("run_after", *phases) - def on_package_attributes(**attr_dict): """Decorator: executes instance function only if object has attr valuses. @@ -475,7 +339,9 @@ def on_package_attributes(**attr_dict): has_all_attributes = all([hasattr(instance, key) for key in attr_dict]) if has_all_attributes: has_the_right_values = all( - [getattr(instance, key) == value for key, value in attr_dict.items()] + [ + getattr(instance, key) == value for key, value in attr_dict.items() + ] # NOQA: ignore=E501 ) if has_the_right_values: func(instance, *args, **kwargs) @@ -687,13 +553,6 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM #: directories, sanity checks will fail. sanity_check_is_dir = [] # type: List[str] - #: List of glob expressions. Each expression must either be - #: absolute or relative to the package source path. - #: Matching artifacts found at the end of the build process will be - #: copied in the same directory tree as _spack_build_logfile and - #: _spack_build_envfile. - archive_files = [] # type: List[str] - #: Boolean. Set to ``True`` for packages that require a manual download. #: This is currently used by package sanity tests and generation of a #: more meaningful fetch failure error. @@ -1038,7 +897,7 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM version (spack.version.Version): the version for which a URL is sought """ uf = None - if type(self).url_for_version != Package.url_for_version: + if type(self).url_for_version != PackageBase.url_for_version: uf = self.url_for_version return self._implement_all_urls_for_version(version, uf) @@ -1959,9 +1818,9 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM even with exceptions. restage (bool): Force spack to restage the package source. skip_patch (bool): Skip patch stage of build if True. - stop_before (InstallPhase): stop execution before this + stop_before (str): stop execution before this installation phase (or None) - stop_at (InstallPhase): last installation phase to be executed + stop_at (str): last installation phase to be executed (or None) tests (bool or list or set): False to run no tests, True to test all packages, or a list of package names to run tests for some @@ -2191,46 +2050,6 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM """ return True - def sanity_check_prefix(self): - """This function checks whether install succeeded.""" - - def check_paths(path_list, filetype, predicate): - if isinstance(path_list, six.string_types): - path_list = [path_list] - - for path in path_list: - abs_path = os.path.join(self.prefix, path) - if not predicate(abs_path): - raise InstallError( - "Install failed for %s. No such %s in prefix: %s" - % (self.name, filetype, path) - ) - - check_paths(self.sanity_check_is_file, "file", os.path.isfile) - check_paths(self.sanity_check_is_dir, "directory", os.path.isdir) - - ignore_file = match_predicate(spack.store.layout.hidden_file_regexes) - if all(map(ignore_file, os.listdir(self.prefix))): - raise InstallError("Install failed for %s. Nothing was installed!" % self.name) - - def apply_macos_rpath_fixups(self): - """On Darwin, make installed libraries more easily relocatable. - - Some build systems (handrolled, autotools, makefiles) can set their own - rpaths that are duplicated by spack's compiler wrapper. This fixup - interrogates, and postprocesses if necessary, all libraries installed - by the code. - - It should be added as a @run_after to packaging systems (or individual - packages) that do not install relocatable libraries by default. - """ - if "platform=darwin" not in self.spec: - return - - from spack.relocate import fixup_macos_rpaths - - fixup_macos_rpaths(self.spec) - @property def build_log_path(self): """ @@ -2268,19 +2087,6 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM """ return None, None, flags - def setup_build_environment(self, env): - """Sets up the build environment for a package. - - This method will be called before the current package prefix exists in - Spack's store. - - Args: - env (spack.util.environment.EnvironmentModifications): environment - modifications to be applied when the package is built. Package authors - can call methods on it to alter the build environment. - """ - pass - def setup_run_environment(self, env): """Sets up the run environment for a package. @@ -2291,37 +2097,6 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM """ pass - def setup_dependent_build_environment(self, env, dependent_spec): - """Sets up the build environment of packages that depend on this one. - - This is similar to ``setup_build_environment``, but it is used to - modify the build environments of packages that *depend* on this one. - - This gives packages like Python and others that follow the extension - model a way to implement common environment or compile-time settings - for dependencies. - - This method will be called before the dependent package prefix exists - in Spack's store. - - Examples: - 1. Installing python modules generally requires ``PYTHONPATH`` - to point to the ``lib/pythonX.Y/site-packages`` directory in the - module's install prefix. This method could be used to set that - variable. - - Args: - env (spack.util.environment.EnvironmentModifications): environment - modifications to be applied when the dependent package is built. - Package authors can call methods on it to alter the build environment. - - dependent_spec (spack.spec.Spec): the spec of the dependent package - about to be built. This allows the extendee (self) to query - the dependent's state. Note that *this* package's spec is - available as ``self.spec`` - """ - pass - def setup_dependent_run_environment(self, env, dependent_spec): """Sets up the run environment of packages that depend on this one. @@ -2508,7 +2283,7 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM def do_uninstall(self, force=False): """Uninstall this package by spec.""" # delegate to instance-less method. - Package.uninstall_by_spec(self.spec, force) + PackageBase.uninstall_by_spec(self.spec, force) def do_deprecate(self, deprecator, link_fn): """Deprecate this package in favor of deprecator spec""" @@ -2560,7 +2335,7 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM deprecated.package.do_deprecate(deprecator, link_fn) # Now that we've handled metadata, uninstall and replace with link - Package.uninstall_by_spec(spec, force=True, deprecator=deprecator) + PackageBase.uninstall_by_spec(spec, force=True, deprecator=deprecator) link_fn(deprecator.prefix, spec.prefix) def _check_extendable(self): @@ -2811,21 +2586,25 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM """ return " ".join("-Wl,-rpath,%s" % p for p in self.rpath) - def _run_test_callbacks(self, method_names, callback_type="install"): + @property + def builder(self): + return spack.builder.create(self) + + @staticmethod + def run_test_callbacks(builder, method_names, callback_type="install"): """Tries to call all of the listed methods, returning immediately if the list is None.""" - if method_names is None: + if not builder.pkg.run_tests or method_names is None: return fail_fast = spack.config.get("config:fail_fast", False) - - with self._setup_test(verbose=False, externals=False) as logger: + with builder.pkg._setup_test(verbose=False, externals=False) as logger: # Report running each of the methods in the build log print_test_message(logger, "Running {0}-time tests".format(callback_type), True) for name in method_names: try: - fn = getattr(self, name) + fn = getattr(builder, name) msg = ("RUN-TESTS: {0}-time tests [{1}]".format(callback_type, name),) print_test_message(logger, msg, True) @@ -2835,27 +2614,13 @@ class PackageBase(six.with_metaclass(PackageMeta, WindowsRPathMeta, PackageViewM msg = ("RUN-TESTS: method not implemented [{0}]".format(name),) print_test_message(logger, msg, True) - self.test_failures.append((e, msg)) + builder.pkg.test_failures.append((e, msg)) if fail_fast: break # Raise any collected failures here - if self.test_failures: - raise TestFailure(self.test_failures) - - @on_package_attributes(run_tests=True) - def _run_default_build_time_test_callbacks(self): - """Tries to call all the methods that are listed in the attribute - ``build_time_test_callbacks`` if ``self.run_tests is True``. - """ - self._run_test_callbacks(self.build_time_test_callbacks, "build") - - @on_package_attributes(run_tests=True) - def _run_default_install_time_test_callbacks(self): - """Tries to call all the methods that are listed in the attribute - ``install_time_test_callbacks`` if ``self.run_tests is True``. - """ - self._run_test_callbacks(self.install_time_test_callbacks, "install") + if builder.pkg.test_failures: + raise TestFailure(builder.pkg.test_failures) def has_test_method(pkg): @@ -2979,37 +2744,6 @@ env_flags = PackageBase.env_flags build_system_flags = PackageBase.build_system_flags -class BundlePackage(PackageBase): - """General purpose bundle, or no-code, package class.""" - - #: There are no phases by default but the property is required to support - #: post-install hooks (e.g., for module generation). - phases = [] # type: List[str] - #: This attribute is used in UI queries that require to know which - #: build-system class we are using - build_system_class = "BundlePackage" - - #: Bundle packages do not have associated source or binary code. - has_code = False - - -class Package(PackageBase): - """General purpose class with a single ``install`` - phase that needs to be coded by packagers. - """ - - #: The one and only phase - phases = ["install"] - #: This attribute is used in UI queries that require to know which - #: build-system class we are using - build_system_class = "Package" - # This will be used as a registration decorator in user - # packages, if need be - run_after("install")(PackageBase.sanity_check_prefix) - # On macOS, force rpaths for shared library IDs and remove duplicate rpaths - run_after("install")(PackageBase.apply_macos_rpath_fixups) - - def install_dependency_symlinks(pkg, spec, prefix): """ Execute a dummy install and flatten dependencies. diff --git a/lib/spack/spack/patch.py b/lib/spack/spack/patch.py index 553b04fda9..06841fa73f 100644 --- a/lib/spack/spack/patch.py +++ b/lib/spack/spack/patch.py @@ -352,7 +352,7 @@ class PatchCache(object): Arguments: sha256 (str): sha256 hash to look up - pkg (spack.package_base.Package): Package object to get patch for. + pkg (spack.package_base.PackageBase): Package object to get patch for. We build patch objects lazily because building them requires that we have information about the package's location in its repo. diff --git a/lib/spack/spack/test/audit.py b/lib/spack/spack/test/audit.py index eded1a92f2..bc5dd3edca 100644 --- a/lib/spack/spack/test/audit.py +++ b/lib/spack/spack/test/audit.py @@ -22,7 +22,7 @@ import spack.config # This package has a GitHub patch URL without full_index=1 (["invalid-github-patch-url"], ["PKG-DIRECTIVES", "PKG-PROPERTIES"]), # This package has a stand-alone 'test' method in build-time callbacks - (["test-build-callbacks"], ["PKG-DIRECTIVES", "PKG-PROPERTIES"]), + (["fail-test-audit"], ["PKG-DIRECTIVES", "PKG-PROPERTIES"]), # This package has no issues (["mpileaks"], None), # This package has a conflict with a trigger which cannot constrain the constraint diff --git a/lib/spack/spack/test/build_systems.py b/lib/spack/spack/test/build_systems.py index f660596bb7..c4255eef49 100644 --- a/lib/spack/spack/test/build_systems.py +++ b/lib/spack/spack/test/build_systems.py @@ -13,10 +13,11 @@ import pytest import llnl.util.filesystem as fs import spack.build_systems.autotools +import spack.build_systems.cmake import spack.environment import spack.platforms import spack.repo -from spack.build_environment import ChildError, get_std_cmake_args, setup_package +from spack.build_environment import ChildError, setup_package from spack.spec import Spec from spack.util.executable import which @@ -144,7 +145,7 @@ class TestAutotoolsPackage(object): # Assert the libtool archive is not there and we have # a log of removed files - assert not os.path.exists(s.package.libtool_archive_file) + assert not os.path.exists(s.package.builder.libtool_archive_file) search_directory = os.path.join(s.prefix, ".spack") libtool_deletion_log = fs.find(search_directory, "removed_la_files.txt", recursive=True) assert libtool_deletion_log @@ -155,11 +156,11 @@ class TestAutotoolsPackage(object): # Install a package that creates a mock libtool archive, # patch its package to preserve the installation s = Spec("libtool-deletion").concretized() - monkeypatch.setattr(s.package, "install_libtool_archives", True) + monkeypatch.setattr(type(s.package.builder), "install_libtool_archives", True) s.package.do_install(explicit=True) # Assert libtool archives are installed - assert os.path.exists(s.package.libtool_archive_file) + assert os.path.exists(s.package.builder.libtool_archive_file) def test_autotools_gnuconfig_replacement(self, mutable_database): """ @@ -253,22 +254,23 @@ class TestCMakePackage(object): def test_cmake_std_args(self): # Call the function on a CMakePackage instance s = Spec("cmake-client").concretized() - assert s.package.std_cmake_args == get_std_cmake_args(s.package) + expected = spack.build_systems.cmake.CMakeBuilder.std_args(s.package) + assert s.package.builder.std_cmake_args == expected # Call it on another kind of package s = Spec("mpich").concretized() - assert get_std_cmake_args(s.package) + assert spack.build_systems.cmake.CMakeBuilder.std_args(s.package) - def test_cmake_bad_generator(self): + def test_cmake_bad_generator(self, monkeypatch): s = Spec("cmake-client").concretized() - s.package.generator = "Yellow Sticky Notes" + monkeypatch.setattr(type(s.package), "generator", "Yellow Sticky Notes", raising=False) with pytest.raises(spack.package_base.InstallError): - get_std_cmake_args(s.package) + s.package.builder.std_cmake_args def test_cmake_secondary_generator(config, mock_packages): s = Spec("cmake-client").concretized() s.package.generator = "CodeBlocks - Unix Makefiles" - assert get_std_cmake_args(s.package) + assert s.package.builder.std_cmake_args def test_define(self): s = Spec("cmake-client").concretized() @@ -361,7 +363,7 @@ def test_autotools_args_from_conditional_variant(config, mock_packages): is not met. When this is the case, the variant is not set in the spec.""" s = Spec("autotools-conditional-variants-test").concretized() assert "example" not in s.variants - assert len(s.package._activate_or_not("example", "enable", "disable")) == 0 + assert len(s.package.builder._activate_or_not("example", "enable", "disable")) == 0 def test_autoreconf_search_path_args_multiple(config, mock_packages, tmpdir): diff --git a/lib/spack/spack/test/builder.py b/lib/spack/spack/test/builder.py new file mode 100644 index 0000000000..bda72c8b49 --- /dev/null +++ b/lib/spack/spack/test/builder.py @@ -0,0 +1,123 @@ +# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other +# Spack Project Developers. See the top-level COPYRIGHT file for details. +# +# SPDX-License-Identifier: (Apache-2.0 OR MIT) +import os.path + +import pytest + +import spack.paths + + +@pytest.fixture() +def builder_test_repository(): + builder_test_path = os.path.join(spack.paths.repos_path, "builder.test") + with spack.repo.use_repositories(builder_test_path) as mock_repo: + yield mock_repo + + +@pytest.mark.parametrize( + "spec_str,expected_values", + [ + ( + "callbacks@2.0", + [ + ("BEFORE_INSTALL_1_CALLED", "1"), + ("BEFORE_INSTALL_2_CALLED", "1"), + ("CALLBACKS_INSTALL_CALLED", "1"), + ("AFTER_INSTALL_1_CALLED", "1"), + ("TEST_VALUE", "3"), + ("INSTALL_VALUE", "CALLBACKS"), + ], + ), + # The last callback is conditional on "@1.0", check it's being executed + ( + "callbacks@1.0", + [ + ("BEFORE_INSTALL_1_CALLED", "1"), + ("BEFORE_INSTALL_2_CALLED", "1"), + ("CALLBACKS_INSTALL_CALLED", "1"), + ("AFTER_INSTALL_1_CALLED", "1"), + ("AFTER_INSTALL_2_CALLED", "1"), + ("TEST_VALUE", "4"), + ("INSTALL_VALUE", "CALLBACKS"), + ], + ), + # The package below adds to "callbacks" using inheritance, test that using super() + # works with builder hierarchies + ( + "inheritance@1.0", + [ + ("DERIVED_BEFORE_INSTALL_CALLED", "1"), + ("BEFORE_INSTALL_1_CALLED", "1"), + ("BEFORE_INSTALL_2_CALLED", "1"), + ("CALLBACKS_INSTALL_CALLED", "1"), + ("INHERITANCE_INSTALL_CALLED", "1"), + ("AFTER_INSTALL_1_CALLED", "1"), + ("AFTER_INSTALL_2_CALLED", "1"), + ("TEST_VALUE", "4"), + ("INSTALL_VALUE", "INHERITANCE"), + ], + ), + # Generate custom phases using a GenericBuilder + ( + "custom-phases", + [ + ("CONFIGURE_CALLED", "1"), + ("INSTALL_CALLED", "1"), + ("LAST_PHASE", "INSTALL"), + ], + ), + # Old-style package, with phase defined in base builder + ( + "old-style-autotools@1.0", + [ + ("AFTER_AUTORECONF_1_CALLED", "1"), + ], + ), + ( + "old-style-autotools@2.0", + [ + ("AFTER_AUTORECONF_2_CALLED", "1"), + ], + ), + ( + "old-style-custom-phases", + [ + ("AFTER_CONFIGURE_CALLED", "1"), + ("TEST_VALUE", "0"), + ], + ), + ], +) +@pytest.mark.usefixtures("builder_test_repository", "config") +@pytest.mark.disable_clean_stage_check +def test_callbacks_and_installation_procedure(spec_str, expected_values, working_env): + """Test the correct execution of callbacks and installation procedures for packages.""" + s = spack.spec.Spec(spec_str).concretized() + builder = spack.builder.create(s.package) + for phase_fn in builder: + phase_fn.execute() + + # Check calls have produced the expected side effects + for var_name, expected in expected_values: + assert os.environ[var_name] == expected, os.environ + + +@pytest.mark.usefixtures("builder_test_repository", "config") +@pytest.mark.parametrize( + "spec_str,method_name,expected", + [ + # Call a function defined on the package, which calls the same function defined + # on the super(builder) + ("old-style-autotools", "configure_args", ["--with-foo"]), + # Call a function defined on the package, which calls the same function defined on the + # super(pkg), which calls the same function defined in the super(builder) + ("old-style-derived", "configure_args", ["--with-bar", "--with-foo"]), + ], +) +def test_old_style_compatibility_with_super(spec_str, method_name, expected): + s = spack.spec.Spec(spec_str).concretized() + builder = spack.builder.create(s.package) + value = getattr(builder, method_name)() + assert value == expected diff --git a/lib/spack/spack/test/cmd/info.py b/lib/spack/spack/test/cmd/info.py index 55b875e022..2bbf4d3a7f 100644 --- a/lib/spack/spack/test/cmd/info.py +++ b/lib/spack/spack/test/cmd/info.py @@ -43,7 +43,7 @@ def test_it_just_runs(pkg): def test_info_noversion(mock_packages, print_buffer): - """Check that a mock package with no versions or variants outputs None.""" + """Check that a mock package with no versions outputs None.""" info("noversion") line_iter = iter(print_buffer) @@ -52,7 +52,7 @@ def test_info_noversion(mock_packages, print_buffer): has = [desc in line for desc in ["Preferred", "Safe", "Deprecated"]] if not any(has): continue - elif "Variants" not in line: + else: continue assert "None" in next(line_iter).strip() diff --git a/lib/spack/spack/test/cmd/install.py b/lib/spack/spack/test/cmd/install.py index 8c37ea3e98..41de6565a1 100644 --- a/lib/spack/spack/test/cmd/install.py +++ b/lib/spack/spack/test/cmd/install.py @@ -1087,7 +1087,7 @@ def test_install_empty_env( ("test-install-callbacks", "undefined-install-test"), ], ) -def test_install_callbacks_fail(install_mockery, mock_fetch, name, method): +def test_installation_fail_tests(install_mockery, mock_fetch, name, method): output = install("--test=root", "--no-cache", name, fail_on_error=False) assert output.count(method) == 2 diff --git a/lib/spack/spack/test/cmd/test.py b/lib/spack/spack/test/cmd/test.py index e842dbc465..fac639309d 100644 --- a/lib/spack/spack/test/cmd/test.py +++ b/lib/spack/spack/test/cmd/test.py @@ -237,8 +237,7 @@ def test_test_list_all(mock_packages): "simple-standalone-test", "test-error", "test-fail", - "test-build-callbacks", - "test-install-callbacks", + "fail-test-audit", ] ) diff --git a/lib/spack/spack/test/concretize.py b/lib/spack/spack/test/concretize.py index 4d15d48dbe..ccdc19a174 100644 --- a/lib/spack/spack/test/concretize.py +++ b/lib/spack/spack/test/concretize.py @@ -228,7 +228,7 @@ class TestConcretize(object): check_concretize(spec) def test_concretize_mention_build_dep(self): - spec = check_concretize("cmake-client ^cmake@3.4.3") + spec = check_concretize("cmake-client ^cmake@3.21.3") # Check parent's perspective of child to_dependencies = spec.edges_to_dependencies(name="cmake") diff --git a/lib/spack/spack/test/concretize_requirements.py b/lib/spack/spack/test/concretize_requirements.py index 50cc6baa9a..e43203b969 100644 --- a/lib/spack/spack/test/concretize_requirements.py +++ b/lib/spack/spack/test/concretize_requirements.py @@ -7,6 +7,7 @@ import sys import pytest +import spack.build_systems.generic import spack.config import spack.repo import spack.util.spack_yaml as syaml @@ -102,7 +103,7 @@ def fake_installs(monkeypatch, tmpdir): stage_path = str(tmpdir.ensure("fake-stage", dir=True)) universal_unused_stage = spack.stage.DIYStage(stage_path) monkeypatch.setattr( - spack.package_base.Package, "_make_stage", MakeStage(universal_unused_stage) + spack.build_systems.generic.Package, "_make_stage", MakeStage(universal_unused_stage) ) diff --git a/lib/spack/spack/test/config.py b/lib/spack/spack/test/config.py index f4d0b570fd..f5f91f039d 100644 --- a/lib/spack/spack/test/config.py +++ b/lib/spack/spack/test/config.py @@ -18,6 +18,7 @@ from llnl.util.filesystem import getuid, join_path, mkdirp, touch, touchp import spack.config import spack.environment as ev import spack.main +import spack.package_base import spack.paths import spack.repo import spack.schema.compilers @@ -1170,13 +1171,13 @@ def test_license_dir_config(mutable_config, mock_packages): """Ensure license directory is customizable""" expected_dir = spack.paths.default_license_dir assert spack.config.get("config:license_dir") == expected_dir - assert spack.package.Package.global_license_dir == expected_dir + assert spack.package_base.PackageBase.global_license_dir == expected_dir assert spack.repo.path.get_pkg_class("a").global_license_dir == expected_dir rel_path = os.path.join(os.path.sep, "foo", "bar", "baz") spack.config.set("config:license_dir", rel_path) assert spack.config.get("config:license_dir") == rel_path - assert spack.package.Package.global_license_dir == rel_path + assert spack.package_base.PackageBase.global_license_dir == rel_path assert spack.repo.path.get_pkg_class("a").global_license_dir == rel_path diff --git a/lib/spack/spack/test/graph.py b/lib/spack/spack/test/graph.py index e7aafe4b0d..60d041d60b 100644 --- a/lib/spack/spack/test/graph.py +++ b/lib/spack/spack/test/graph.py @@ -104,6 +104,19 @@ o dyninst |/ o libelf """ + or graph_str + == r"""o mpileaks +|\ +o | callpath +|\| +| o mpich +| +o dyninst +|\ +o | libdwarf +|/ +o libelf +""" ) diff --git a/lib/spack/spack/test/install.py b/lib/spack/spack/test/install.py index 9093593bed..e8b6f415d3 100644 --- a/lib/spack/spack/test/install.py +++ b/lib/spack/spack/test/install.py @@ -144,13 +144,12 @@ class MockStage(object): return getattr(self.wrapped_stage, attr) -def test_partial_install_delete_prefix_and_stage(install_mockery, mock_fetch): +def test_partial_install_delete_prefix_and_stage(install_mockery, mock_fetch, working_env): s = Spec("canfail").concretized() instance_rm_prefix = s.package.remove_prefix try: - s.package.succeed = False s.package.remove_prefix = mock_remove_prefix with pytest.raises(MockInstallError): s.package.do_install() @@ -161,7 +160,7 @@ def test_partial_install_delete_prefix_and_stage(install_mockery, mock_fetch): # must clear failure markings for the package before re-installing it spack.store.db.clear_failure(s, True) - s.package.succeed = True + s.package.set_install_succeed() s.package.stage = MockStage(s.package.stage) s.package.do_install(restage=True) @@ -174,18 +173,20 @@ def test_partial_install_delete_prefix_and_stage(install_mockery, mock_fetch): @pytest.mark.disable_clean_stage_check -def test_failing_overwrite_install_should_keep_previous_installation(mock_fetch, install_mockery): +def test_failing_overwrite_install_should_keep_previous_installation( + mock_fetch, install_mockery, working_env +): """ Make sure that whenever `spack install --overwrite` fails, spack restores the original install prefix instead of cleaning it. """ # Do a successful install s = Spec("canfail").concretized() - s.package.succeed = True + s.package.set_install_succeed() # Do a failing overwrite install s.package.do_install() - s.package.succeed = False + s.package.set_install_fail() kwargs = {"overwrite": [s.dag_hash()]} with pytest.raises(Exception): @@ -238,13 +239,11 @@ def test_install_dependency_symlinks_pkg(install_mockery, mock_fetch, mutable_mo def test_install_times(install_mockery, mock_fetch, mutable_mock_repo): """Test install times added.""" - spec = Spec("dev-build-test-install-phases") - spec.concretize() - pkg = spec.package - pkg.do_install() + spec = Spec("dev-build-test-install-phases").concretized() + spec.package.do_install() # Ensure dependency directory exists after the installation. - install_times = os.path.join(pkg.prefix, ".spack", "install_times.json") + install_times = os.path.join(spec.package.prefix, ".spack", "install_times.json") assert os.path.isfile(install_times) # Ensure the phases are included @@ -346,12 +345,11 @@ def test_installed_upstream(install_upstream, mock_fetch): @pytest.mark.disable_clean_stage_check -def test_partial_install_keep_prefix(install_mockery, mock_fetch, monkeypatch): +def test_partial_install_keep_prefix(install_mockery, mock_fetch, monkeypatch, working_env): s = Spec("canfail").concretized() # If remove_prefix is called at any point in this test, that is an error - s.package.succeed = False # make the build fail - monkeypatch.setattr(spack.package_base.Package, "remove_prefix", mock_remove_prefix) + monkeypatch.setattr(spack.package_base.PackageBase, "remove_prefix", mock_remove_prefix) with pytest.raises(spack.build_environment.ChildError): s.package.do_install(keep_prefix=True) assert os.path.exists(s.package.prefix) @@ -359,7 +357,7 @@ def test_partial_install_keep_prefix(install_mockery, mock_fetch, monkeypatch): # must clear failure markings for the package before re-installing it spack.store.db.clear_failure(s, True) - s.package.succeed = True # make the build succeed + s.package.set_install_succeed() s.package.stage = MockStage(s.package.stage) s.package.do_install(keep_prefix=True) assert s.package.spec.installed @@ -368,14 +366,14 @@ def test_partial_install_keep_prefix(install_mockery, mock_fetch, monkeypatch): def test_second_install_no_overwrite_first(install_mockery, mock_fetch, monkeypatch): s = Spec("canfail").concretized() - monkeypatch.setattr(spack.package_base.Package, "remove_prefix", mock_remove_prefix) + monkeypatch.setattr(spack.package_base.PackageBase, "remove_prefix", mock_remove_prefix) - s.package.succeed = True + s.package.set_install_succeed() s.package.do_install() assert s.package.spec.installed # If Package.install is called after this point, it will fail - s.package.succeed = False + s.package.set_install_fail() s.package.do_install() @@ -589,7 +587,9 @@ def test_log_install_with_build_files(install_mockery, monkeypatch): source = spec.package.stage.source_path config = os.path.join(source, "config.log") fs.touchp(config) - spec.package.archive_files = ["missing", "..", config] + monkeypatch.setattr( + type(spec.package), "archive_files", ["missing", "..", config], raising=False + ) spack.installer.log(spec.package) diff --git a/lib/spack/spack/test/repo.py b/lib/spack/spack/test/repo.py index fcd009edec..402723d226 100644 --- a/lib/spack/spack/test/repo.py +++ b/lib/spack/spack/test/repo.py @@ -113,14 +113,14 @@ def test_absolute_import_spack_packages_as_python_modules(mock_packages): assert hasattr(spack.pkg.builtin.mock, "mpileaks") assert hasattr(spack.pkg.builtin.mock.mpileaks, "Mpileaks") assert isinstance(spack.pkg.builtin.mock.mpileaks.Mpileaks, spack.package_base.PackageMeta) - assert issubclass(spack.pkg.builtin.mock.mpileaks.Mpileaks, spack.package_base.Package) + assert issubclass(spack.pkg.builtin.mock.mpileaks.Mpileaks, spack.package_base.PackageBase) def test_relative_import_spack_packages_as_python_modules(mock_packages): from spack.pkg.builtin.mock.mpileaks import Mpileaks assert isinstance(Mpileaks, spack.package_base.PackageMeta) - assert issubclass(Mpileaks, spack.package_base.Package) + assert issubclass(Mpileaks, spack.package_base.PackageBase) def test_all_virtual_packages_have_default_providers(): diff --git a/lib/spack/spack/util/package_hash.py b/lib/spack/spack/util/package_hash.py index 4877748338..f4435fbba4 100644 --- a/lib/spack/spack/util/package_hash.py +++ b/lib/spack/spack/util/package_hash.py @@ -64,7 +64,7 @@ class RemoveDirectives(ast.NodeTransformer): # list of URL attributes and metadata attributes # these will be removed from packages. self.metadata_attrs = [s.url_attr for s in spack.fetch_strategy.all_strategies] - self.metadata_attrs += spack.package_base.Package.metadata_attrs + self.metadata_attrs += spack.package_base.PackageBase.metadata_attrs self.spec = spec self.in_classdef = False # used to avoid nested classdefs @@ -158,6 +158,7 @@ class TagMultiMethods(ast.NodeVisitor): def visit_FunctionDef(self, func): conditions = [] + for dec in func.decorator_list: if isinstance(dec, ast.Call) and dec.func.id == "when": try: diff --git a/lib/spack/spack/util/web.py b/lib/spack/spack/util/web.py index fa4119f917..939ec669c0 100644 --- a/lib/spack/spack/util/web.py +++ b/lib/spack/spack/util/web.py @@ -784,7 +784,7 @@ def find_versions_of_archive( list_depth (int): max depth to follow links on list_url pages. Defaults to 0. concurrency (int): maximum number of concurrent requests - reference_package (spack.package_base.Package or None): a spack package + reference_package (spack.package_base.PackageBase or None): a spack package used as a reference for url detection. Uses the url_for_version method on the package to produce reference urls which, if found, are preferred. diff --git a/lib/spack/spack/variant.py b/lib/spack/spack/variant.py index 864ea26446..92e512219b 100644 --- a/lib/spack/spack/variant.py +++ b/lib/spack/spack/variant.py @@ -96,7 +96,7 @@ class Variant(object): Args: vspec (Variant): instance to be validated - pkg_cls (spack.package_base.Package): the package class + pkg_cls (spack.package_base.PackageBase): the package class that required the validation, if available Raises: |
