summaryrefslogtreecommitdiff
path: root/lib
diff options
context:
space:
mode:
authorTodd Gamblin <tgamblin@llnl.gov>2014-01-12 18:22:19 +0100
committerTodd Gamblin <tgamblin@llnl.gov>2014-01-12 18:22:19 +0100
commit91d6f0b4815d57768422a07c1ecc7a743222caf6 (patch)
treecbed0cc09f3b0ae6880ee6726736a0d6387ed70a /lib
parentd18be7437febdecbd6f0ff0bd9101d246db27bab (diff)
downloadspack-91d6f0b4815d57768422a07c1ecc7a743222caf6.tar.gz
spack-91d6f0b4815d57768422a07c1ecc7a743222caf6.tar.bz2
spack-91d6f0b4815d57768422a07c1ecc7a743222caf6.tar.xz
spack-91d6f0b4815d57768422a07c1ecc7a743222caf6.zip
Developer documentation
Diffstat (limited to 'lib')
-rw-r--r--lib/spack/docs/basic_usage.rst2
-rw-r--r--lib/spack/docs/developer_guide.rst274
-rw-r--r--lib/spack/docs/packaging_guide.rst6
-rw-r--r--lib/spack/docs/site_configuration.rst3
-rw-r--r--lib/spack/spack/compilation.py10
-rw-r--r--lib/spack/spack/version.py2
6 files changed, 292 insertions, 5 deletions
diff --git a/lib/spack/docs/basic_usage.rst b/lib/spack/docs/basic_usage.rst
index fc2312134e..4c527d91d2 100644
--- a/lib/spack/docs/basic_usage.rst
+++ b/lib/spack/docs/basic_usage.rst
@@ -1,4 +1,4 @@
-.. _basic_usage:
+.. _basic-usage:
Basic usage
=====================
diff --git a/lib/spack/docs/developer_guide.rst b/lib/spack/docs/developer_guide.rst
index 47b98a211d..1f0977b4de 100644
--- a/lib/spack/docs/developer_guide.rst
+++ b/lib/spack/docs/developer_guide.rst
@@ -3,8 +3,258 @@
Developer Guide
=====================
-This guide is intended for people who want to work on Spack's inner
-workings. Right now it's pretty sparse.
+This guide is intended for people who want to work on Spack itself.
+If you just want to develop pacakges, see the :ref:`packaging-guide`.
+
+It is assumed that you've read the :ref:`basic-usage` and
+:ref:`packaging-guide` sections, and that you're familiar with the
+concepts discussed there. If you're not, we recommend reading those
+first.
+
+Overview
+-----------------------
+
+Spack is designed with three separate roles in mind:
+
+ #. **Users**, who need to install software *without* knowing all the
+ details about how it is built.
+ #. **Packagers** who know how a particular software package is
+ built and encode this information in package files.
+ #. **Developers** who work on Spack, add new features, and try to
+ make the jobs of packagers and users easier.
+
+Users could be end users installing software in their home directory,
+or administrators installing software to a shared directory on a
+shared machine. Packagers could be administrators who want to
+automate software builds, or application developers who want to make
+their software more accessible to users.
+
+As you might expect, there are many types of users with different
+levels of sophistication, and Spack is designed to accommodate both
+simple and complex use cases for packages. A user who only knows that
+he needs a certain package should be able to type something simple,
+like ``spack install <package name>``, and get the package that he
+wants. If a user wants to ask for a specific version, use particular
+compilers, or build several versions with different configurations,
+then that should be possible with a minimal amount of additional
+specification.
+
+This gets us to the two key concepts in Spack's software design:
+
+ #. **Specs**: expressions for describing builds of software, and
+ #. **Packages**: Python modules that build software according to a
+ spec.
+
+A package is a template for building particular software, and a spec
+as a descriptor for one or more instances of that template. Users
+express the configuration they want using a spec, and a package turns
+the spec into a complete build.
+
+The obvious difficulty with this design is that users underspecify
+what they want. To build a software package, the package object needs
+a *complete* specification. In Spack, if a spec describes only one
+instance of a package, then we say it is **concrete**. If a spec
+could describes many instances, (i.e. it is underspecified in one way
+or another), then we say it is **abstract**.
+
+Spack's job is to take an *abstract* spec from the user, find a
+*concrete* spec that satisfies the constraints, and hand the task of
+building the software off to the package object. The rest of this
+document describes all the pieces that come together to make that
+happen.
+
+
+Directory Structure
+-------------------------
+
+So that you can familiarize yourself with the project, we'll start
+with a high level view of Spack's directory structure::
+
+ spack/ <- installation root
+ bin/
+ spack <- main spack executable
+ var/
+ spack/ <- build & stage directories
+ opt/
+ spack/ <- packages are installed here
+ lib/
+ spack/
+ docs/ <- source for this documentation
+ env/ <- compiler wrappers for build environment
+
+ spack/ <- spack module; contains Python code
+ cmd/ <- each file in here is a spack subcommand
+ compilers/ <- compiler description files
+ packages/ <- each file in here is a spack package
+ test/ <- unit test modules
+ util/ <- common code
+
+Spack is designed so that it could live within a `standard UNIX
+directory hierarchy <http://linux.die.net/man/7/hier>`_, so ``lib``,
+``var``, and ``opt`` all contain a ``spack`` subdirectory in case
+Spack is installed alongside other software. Most of the insteresting
+parts of Spack live in ``lib/spack``. Files under ``var`` are created
+as needed, so there is no ``var`` directory when you initially clone
+Spack from the repository.
+
+Spack has *one* directory layout and there is no install process.
+version and the source code. Most Python programs don't look like
+this (they use distutils, ``setup.py``, etc.) but we wanted to make
+Spack *very* easy to use. The simple layout spares users from the
+need to install Spack into a Python environment. Many users don't
+have write access to a Python installation, and installing an entire
+new instance of Python to bootstrap Spack would be very complicated.
+Users should not have to install install a big, complicated package to
+use the thing that's supposed to spare them from the details of big,
+complicated packages. The end result is that Spack works out of the
+box: clone it and add ``bin`` to your PATH and you're ready to go.
+
+
+Code Structure
+-------------------------
+
+This section gives an overview of the various Python modules in Spack,
+grouped by functionality.
+
+Package-related modules
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:mod:`spack.package`
+ Contains the :class:`Package <spack.package.Package>` class, which
+ is the superclass for all packages in Spack. Methods on ``Package``
+ implement all phases of the :ref:`package lifecycle
+ <pacakge-lifecycle>` and manage the build process.
+
+:mod:`spack.packages`
+ Contains all of the packages in Spack and methods for managing them.
+ Functions like :func:`packages.get <spack.packages.get>` and
+ :func:`class_name_for_package_name
+ <packages.class_name_for_package_name>` handle mapping packge module
+ names to class names and dynamically instantiating packages by name
+ from module files.
+
+:mod:`spack.relations`
+ *Relations* are relationships between packages, like
+ :func:`depends_on <spack.relations.depends_on>` and :func:`provides
+ <spack.relations.provides>`. See :ref:`dependencies` and
+ :ref:`virtual-dependencies`.
+
+:mod:`spack.multimethod`
+ Implementation of the :func:`@when <spack.multimethod.when>`
+ decorator, which allows :ref:`multimethods <multimethods>` in
+ packages.
+
+
+Spec-related modules
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:mod:`spack.spec`
+ Contains :class:`Spec <spack.spec.Spec>` and :class:`SpecParser
+ <spack.spec.SpecParser>`. Also implements most of the logic for
+ normalization and concretization of specs.
+
+:mod:`spack.parse`
+ Contains some base classes for implementing simple recursive descent
+ parsers: :class:`Parser <spack.parse.Parser>` and :class:`Lexer
+ <spack.parse.Lexer>`. Used by :class:`SpecParser
+ <spack.spec.SpecParser>`.
+
+:mod:`spack.concretize`
+ Contains :class:`DefaultConcretizer
+ <spack.concretize.DefaultConcretizer>` implementation, which allows
+ site administrators to change Spack's :ref:`concretization-policies`.
+
+:mod:`spack.version`
+ Implements a simple :class:`Version <spack.version.Version>` class
+ with simple comparison semantics. Also implements
+ :class:`VersionRange <spack.version.VersionRange>` and
+ :class:`VersionList <spack.version.VersionList>`. All three are
+ comparable with each other and offer union and intersection
+ operations. Spack uses these classes to compare versions and to
+ manage version constraints on specs. Comparison semantics are
+ similar to the ``LooseVersion`` class in ``distutils`` and to the
+ way RPM compares version strings.
+
+:mod:`spack.compilers`
+ Submodules contains descriptors for all valid compilers in Spack.
+ This is used by the build system to set up the build environment.
+
+ .. warning::
+
+ Not yet implemented. Currently has two compiler descriptions,
+ but compilers aren't fully integrated with the build process
+ yet.
+
+:mod:`spack.architecture`
+ :func:`architecture.sys_type <spack.architecture.sys_type>` is used
+ to determine the host architecture while building.
+
+ .. warning::
+
+ Not yet implemented. Should eventually have architecture
+ descriptions for cross-compiling.
+
+
+Build environment
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:mod:`spack.stage`
+ Handles creating temporary directories for builds.
+
+:mod:`spack.compilation`
+ This contains utility functions used by the compiler wrapper script,
+ ``cc``.
+
+:mod:`spack.directory_layout`
+ Classes that control the way an installation directory is laid out.
+ Create more implementations of this to change the hierarchy and
+ naming scheme in ``$spack_prefix/opt``
+
+Spack Subcommands
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:mod:`spack.cmd`
+ Each module in this package implements a Spack subcommand. See
+ :ref:`writing commands <writing-commands>` for details.
+
+Unit tests
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:mod:`spack.test`
+ Implements Spack's test suite. Add a module and put its name in
+ the test suite in ``__init__.py`` to add more unit tests.
+
+:mod:`spack.test.mock_packages`
+ This is a fake package hierarchy used to mock up packages for
+ Spack's test suite.
+
+Other Modules
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:mod:`spack.globals`
+ Includes global settings for Spack. the default policy classes for
+ things like :ref:`temporary space <temp-space>` and
+ :ref:`concretization <concretization-policies>`.
+
+:mod:`spack.tty`
+ Basic output functions for all of the messages Spack writes to the
+ terminal.
+
+:mod:`spack.color`
+ Implements a color formatting syntax used by ``spack.tty``.
+
+:mod:`spack.url`
+ URL parsing, for deducing names and versions of packages from
+ tarball URLs.
+
+:mod:`spack.util`
+ In this package are a number of utility modules for the rest of
+ Spack.
+
+:mod:`spack.error`
+ :class:`SpackError <spack.error.SpackError>`, the base class for
+ Spack's exception hierarchy.
+
Spec objects
-------------------------
@@ -12,9 +262,29 @@ Spec objects
Package objects
-------------------------
+
+Most spack commands
+look something like this:
+
+ #. Parse an abstract spec (or specs) from the command line,
+ #. *Normalize* the spec based on information in package files,
+ #. *Concretize* the spec according to some customizable policies,
+ #. Instantiate a package based on the spec, and
+ #. Call methods (e.g., ``install()``) on the package object.
+
+
+
+The information in Package files is used at all stages in this
+process.
+
+
+Conceptually, packages are overloaded. They contain
+
Stage objects
-------------------------
+.. _writing-commands:
+
Writing commands
-------------------------
diff --git a/lib/spack/docs/packaging_guide.rst b/lib/spack/docs/packaging_guide.rst
index fa3f5f43e2..39b41f8756 100644
--- a/lib/spack/docs/packaging_guide.rst
+++ b/lib/spack/docs/packaging_guide.rst
@@ -6,7 +6,7 @@ Packaging Guide
This guide is intended for developers or administrators who want to
*package* their software so that Spack can install it. We assume that
you have at least some familiarty with Python, and that you've read
-the :ref:`basic usage guide <basic_usage>`, especially the part
+the :ref:`basic usage guide <basic-usage>`, especially the part
about :ref:`specs <sec-specs>`.
There are two key parts of Spack:
@@ -459,6 +459,8 @@ this:
:end-before: versions
+.. _dependencies:
+
Dependencies
------------------------------
@@ -513,6 +515,7 @@ on the command line to find specs or to install specs with particular
constraints, and package authors can use it to describe relationships
between packages.
+.. _virtual-dependencies:
Virtual dependencies
-----------------------------
@@ -993,6 +996,7 @@ do all the same things you'd do with the package's own spec:
mpicc = new_path(my_mpi.prefix.bin, 'mpicc')
+.. _multimethods:
Multimethods and ``@when``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/lib/spack/docs/site_configuration.rst b/lib/spack/docs/site_configuration.rst
index ae52df2254..2658e0a310 100644
--- a/lib/spack/docs/site_configuration.rst
+++ b/lib/spack/docs/site_configuration.rst
@@ -3,6 +3,8 @@
Site-specific configuration
===================================
+.. _temp-space:
+
Temporary space
----------------------------
@@ -47,6 +49,7 @@ in the first directory it finds to which it has write access. Add
more elements to the list to indicate where your own site's temporary
directory is.
+.. _concretization-policies:
Concretization policies
----------------------------
diff --git a/lib/spack/spack/compilation.py b/lib/spack/spack/compilation.py
index c4561a4122..3a469376a8 100644
--- a/lib/spack/spack/compilation.py
+++ b/lib/spack/spack/compilation.py
@@ -22,6 +22,16 @@
# along with this program; if not, write to the Free Software Foundation,
# Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
+"""\
+The ``compilation`` module contains utility functions used by the compiler
+wrapper script.
+
+.. todo::
+
+ Think about moving this into the script to increase compilation
+ speed.
+
+"""
import os
import sys
diff --git a/lib/spack/spack/version.py b/lib/spack/spack/version.py
index a99a5a7b4d..c433fa8962 100644
--- a/lib/spack/spack/version.py
+++ b/lib/spack/spack/version.py
@@ -23,7 +23,7 @@
# Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
"""
-This file implements Version and version-ish objects. These are:
+This module implements Version and version-ish objects. These are:
Version
A single version of a package.