From 8ce62ba51334f0f9e4b62f795923d81514229013 Mon Sep 17 00:00:00 2001 From: "Adam J. Stewart" Date: Tue, 17 Jul 2018 13:28:38 -0500 Subject: Add documentation on build systems (#5015) Spack provides a number of classes based on commonly-used build systems that users can extend when writing packages; the classes provide functionality to perform the actions relevant to the build system (e.g. running "configure" for an Autotools-based package). This adds documentation for classes supporting the following build systems: * Makefile * Autotools * CMake * QMake * SCons * Waf This includes build systems for managing extensions of the following packages: * Perl * Python * R * Octave This also adds documentation on implementing packages that use a custom build system (e.g. Perl/CMake). Spack also provides extendable classes which aggregate functionality for related sets of packages, e.g. those using CUDA. Documentation is added for CudaPackage. --- lib/spack/docs/build_systems/makefilepackage.rst | 304 +++++++++++++++++++++++ 1 file changed, 304 insertions(+) create mode 100644 lib/spack/docs/build_systems/makefilepackage.rst (limited to 'lib/spack/docs/build_systems/makefilepackage.rst') diff --git a/lib/spack/docs/build_systems/makefilepackage.rst b/lib/spack/docs/build_systems/makefilepackage.rst new file mode 100644 index 0000000000..8a0cd04dcd --- /dev/null +++ b/lib/spack/docs/build_systems/makefilepackage.rst @@ -0,0 +1,304 @@ +.. _makefilepackage: + +--------------- +MakefilePackage +--------------- + +The most primitive build system a package can use is a plain Makefile. +Makefiles are simple to write for small projects, but they usually +require you to edit the Makefile to set platform and compiler-specific +variables. + +^^^^^^ +Phases +^^^^^^ + +The ``MakefilePackage`` base class comes with 3 phases: + +#. ``edit`` - edit the Makefile +#. ``build`` - build the project +#. ``install`` - install the project + +By default, ``edit`` does nothing, but you can override it to replace +hard-coded Makefile variables. The ``build`` and ``install`` phases +run: + +.. code-block:: console + + $ make + $ make install + + +^^^^^^^^^^^^^^^ +Important files +^^^^^^^^^^^^^^^ + +The main file that matters for a ``MakefilePackage`` is the Makefile. +This file will be named one of the following ways: + +* GNUmakefile (only works with GNU Make) +* Makefile (most common) +* makefile + +Some Makefiles also *include* other configuration files. Check for an +``include`` directive in the Makefile. + +^^^^^^^^^^^^^^^^^^^^^^^^^ +Build system dependencies +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Spack assumes that the operating system will have a valid ``make`` utility +installed already, so you don't need to add a dependency on ``make``. +However, if the package uses a ``GNUmakefile`` or the developers recommend +using GNU Make, you should add a dependency on ``gmake``: + +.. code-block:: python + + depends_on('gmake', type='build') + + +^^^^^^^^^^^^^^^^^^^^^^^^^^ +Types of Makefile packages +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Most of the work involved in packaging software that uses Makefiles +involves overriding or replacing hard-coded variables. Many packages +make the mistake of hard-coding compilers, usually for GCC or Intel. +This is fine if you happen to be using that particular compiler, but +Spack is designed to work with *any* compiler, and you need to ensure +that this is the case. + +Depending on how the Makefile is designed, there are 4 common strategies +that can be used to set or override the appropriate variables: + +""""""""""""""""""""" +Environment variables +""""""""""""""""""""" + +Make has multiple types of +`assignment operators `_. +Some Makefiles use ``=`` to assign variables. The only way to override +these variables is to edit the Makefile or override them on the +command-line. However, Makefiles that use ``?=`` for assignment honor +environment variables. Since Spack already sets ``CC``, ``CXX``, ``F77``, +and ``FC``, you won't need to worry about setting these variables. If +there are any other variables you need to set, you can do this in the +``edit`` method: + +.. code-block:: python + + def edit(self, spec, prefix): + env['PREFIX'] = prefix + env['BLASLIB'] = spec['blas'].libs.ld_flags + + +`cbench `_ +is a good example of a simple package that does this, while +`esmf `_ +is a good example of a more complex package. + +"""""""""""""""""""""" +Command-line arguments +"""""""""""""""""""""" + +If the Makefile ignores environment variables, the next thing to try +is command-line arguments. You can do this by overriding the +``build_targets`` attribute. If you don't need access to the spec, +you can do this like so: + +.. code-block:: python + + build_targets = ['CC=cc'] + + +If you do need access to the spec, you can create a property like so: + +.. code-block:: python + + @property + def build_targets(self): + spec = self.spec + + return [ + 'CC=cc', + 'BLASLIB={0}'.format(spec['blas'].libs.ld_flags), + ] + + +`cloverleaf `_ +is a good example of a package that uses this strategy. + +""""""""""""" +Edit Makefile +""""""""""""" + +Some Makefiles are just plain stubborn and will ignore command-line +variables. The only way to ensure that these packages build correctly +is to directly edit the Makefile. Spack provides a ``FileFilter`` class +and a ``filter_file`` method to help with this. For example: + +.. code-block:: python + + def edit(self, spec, prefix): + makefile = FileFilter('Makefile') + + makefile.filter('CC = gcc', 'CC = cc') + makefile.filter('CXX = g++', 'CC = c++') + + +`stream `_ +is a good example of a package that involves editing a Makefile to set +the appropriate variables. + +""""""""""" +Config file +""""""""""" + +More complex packages often involve Makefiles that *include* a +configuration file. These configuration files are primarily composed +of variables relating to the compiler, platform, and the location of +dependencies or names of libraries. Since these config files are +dependent on the compiler and platform, you will often see entire +directories of examples for common compilers and architectures. Use +these examples to help determine what possible values to use. + +If the config file is long and only contains one or two variables +that need to be modified, you can use the technique above to edit +the config file. However, if you end up needing to modify most of +the variables, it may be easier to write a new file from scratch. + +If each variable is independent of each other, a dictionary works +well for storing variables: + +.. code-block:: python + + def edit(self, spec, prefix): + config = { + 'CC': 'cc', + 'MAKE': 'make', + } + + if '+blas' in spec: + config['BLAS_LIBS'] = spec['blas'].libs.joined() + + with open('make.inc', 'w') as inc: + for key in config: + inc.write('{0} = {1}\n'.format(key, config[key])) + + +`elk `_ +is a good example of a package that uses a dictionary to store +configuration variables. + +If the order of variables is important, it may be easier to store +them in a list: + +.. code-block:: python + + def edit(self, spec, prefix): + config = [ + 'INSTALL_DIR = {0}'.format(prefix), + 'INCLUDE_DIR = $(INSTALL_DIR)/include', + 'LIBRARY_DIR = $(INSTALL_DIR)/lib', + ] + + with open('make.inc', 'w') as inc: + for var in config: + inc.write('{0}\n'.format(var)) + + +`hpl `_ +is a good example of a package that uses a list to store +configuration variables. + +^^^^^^^^^^^^^^^^^^^^^^^^^^ +Variables to watch out for +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The following is a list of common variables to watch out for. The first +two sections are +`implicit variables `_ +defined by Make and will always use the same name, while the rest are +user-defined variables and may vary from package to package. + +* **Compilers** + + This includes variables such as ``CC``, ``CXX``, ``F77``, ``F90``, + and ``FC``, as well as variables related to MPI compiler wrappers, + like ``MPICC`` and friends. + +* **Compiler flags** + + This includes variables for specific compilers, like ``CFLAGS``, + ``CXXFLAGS``, ``F77FLAGS``, ``F90FLAGS``, ``FCFLAGS``, and ``CPPFLAGS``. + These variables are often hard-coded to contain flags specific to a + certain compiler. If these flags don't work for every compiler, + you may want to consider filtering them. + +* **Variables that enable or disable features** + + This includes variables like ``MPI``, ``OPENMP``, ``PIC``, and + ``DEBUG``. These flags often require you to create a variant + so that you can either build with or without MPI support, for + example. These flags are often compiler-dependent. You should + replace them with the appropriate compiler flags, such as + ``self.compiler.openmp_flag`` or ``self.compiler.pic_flag``. + +* **Platform flags** + + These flags control the type of architecture that the executable + is compiler for. Watch out for variables like ``PLAT`` or ``ARCH``. + +* **Dependencies** + + Look out for variables that sound like they could be used to + locate dependencies, such as ``JAVA_HOME``, ``JPEG_ROOT``, or + ``ZLIBDIR``. Also watch out for variables that control linking, + such as ``LIBS``, ``LDFLAGS``, and ``INCLUDES``. These variables + need to be set to the installation prefix of a dependency, or + to the correct linker flags to link to that dependency. + +* **Installation prefix** + + If your Makefile has an ``install`` target, it needs some way of + knowing where to install. By default, many packages install to + ``/usr`` or ``/usr/local``. Since many Spack users won't have + sudo privileges, it is imperative that each package is installed + to the proper prefix. Look for variables like ``PREFIX`` or + ``INSTALL``. + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Makefiles in a sub-directory +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Not every package places their Makefile in the root of the package +tarball. If the Makefile is in a sub-directory like ``src``, you +can tell Spack where to locate it like so: + +.. code-block:: python + + build_directory = 'src' + + +^^^^^^^^^^^^^^^^^^^ +Manual installation +^^^^^^^^^^^^^^^^^^^ + +Not every Makefile includes an ``install`` target. If this is the +case, you can override the default ``install`` method to manually +install the package: + +.. code-block:: python + + def install(self, spec, prefix): + mkdir(prefix.bin) + install('foo', prefix.bin) + install_tree('lib', prefix.lib) + + +^^^^^^^^^^^^^^^^^^^^^^ +External documentation +^^^^^^^^^^^^^^^^^^^^^^ + +For more information on reading and writing Makefiles, see: +https://www.gnu.org/software/make/manual/make.html -- cgit v1.2.3-60-g2f50