summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--lib/spack/docs/getting_started.rst101
-rw-r--r--lib/spack/docs/module_file_support.rst12
-rw-r--r--lib/spack/spack/cmd/bootstrap.py125
-rw-r--r--lib/spack/spack/cmd/clone.py107
-rwxr-xr-xshare/spack/setup-env.sh57
-rw-r--r--var/spack/repos/builtin/packages/environment-modules/package.py5
6 files changed, 292 insertions, 115 deletions
diff --git a/lib/spack/docs/getting_started.rst b/lib/spack/docs/getting_started.rst
index ef0f25cb04..ffa9bf2414 100644
--- a/lib/spack/docs/getting_started.rst
+++ b/lib/spack/docs/getting_started.rst
@@ -52,7 +52,7 @@ For a richer experience, use Spack's shell support:
.. code-block:: console
- # For bash users
+ # For bash/zsh users
$ export SPACK_ROOT=/path/to/spack
$ . $SPACK_ROOT/share/spack/setup-env.sh
@@ -60,10 +60,15 @@ For a richer experience, use Spack's shell support:
$ setenv SPACK_ROOT /path/to/spack
$ source $SPACK_ROOT/share/spack/setup-env.csh
+
This automatically adds Spack to your ``PATH`` and allows the ``spack``
-command to :ref:`load environment modules <shell-support>` and execute
+command to be used to execute spack :ref:`commands <shell-support>` and
:ref:`useful packaging commands <packaging-shell-support>`.
+If :ref:`environment-modules or dotkit <InstallEnvironmentModules>` is
+installed and available, the ``spack`` command can also load and unload
+:ref:`modules <modules>`.
+
^^^^^^^^^^^^^^^^^
Clean Environment
^^^^^^^^^^^^^^^^^
@@ -94,12 +99,12 @@ Optional: Alternate Prefix
^^^^^^^^^^^^^^^^^^^^^^^^^^
You may want to run Spack out of a prefix other than the git repository
-you cloned. The ``spack bootstrap`` command provides this
+you cloned. The ``spack clone`` command provides this
functionality. To install spack in a new directory, simply type:
.. code-block:: console
- $ spack bootstrap /my/favorite/prefix
+ $ spack clone /my/favorite/prefix
This will install a new spack script in ``/my/favorite/prefix/bin``,
which you can use just like you would the regular spack script. Each
@@ -849,6 +854,10 @@ well. They can generally be activated as in the ``curl`` example above;
or some systems might already have an appropriate hand-built
environment module that may be loaded. Either way works.
+If you find that you are missing some of these programs, ``spack`` can
+build some of them for you with ``spack bootstrap``. Currently supported
+programs are ``environment-modules``.
+
A few notes on specific programs in this list:
""""""""""""""""""""""""""
@@ -885,52 +894,72 @@ Environment Modules
In order to use Spack's generated environment modules, you must have
installed one of *Environment Modules* or *Lmod*. On many Linux
distributions, this can be installed from the vendor's repository. For
-example: ``yum install environment-modules`` (Fedora/RHEL/CentOS). If
-your Linux distribution does not have Environment Modules, you can get it
-with Spack:
+example: ``yum install environment-modules`` (Fedora/RHEL/CentOS). If
+your Linux distribution does not have Environment Modules, Spack can
+build it for you!
-#. Consider using system tcl (as long as your system has Tcl version 8.0 or later):
+What follows are three steps describing how to install and use environment-modules with spack.
- #) Identify its location using ``which tclsh``
- #) Identify its version using ``echo 'puts $tcl_version;exit 0' | tclsh``
- #) Add to ``~/.spack/packages.yaml`` and modify as appropriate:
+#. Install ``environment-modules``.
- .. code-block:: yaml
+ * ``spack bootstrap`` will build ``environment-modules`` for you (and may build
+ other packages that are useful to the operation of Spack)
- packages:
- tcl:
- paths:
- tcl@8.5: /usr
- buildable: False
+ * Install ``environment-modules`` using ``spack install`` with
+ ``spack install environment-modules~X`` (The ``~X`` variant builds without Xorg
+ dependencies, but ``environment-modules`` works fine too.)
-#. Install with:
+#. Add ``modulecmd`` to ``PATH`` and create a ``module`` command.
- .. code-block:: console
+ * If you are using ``bash`` or ``ksh``, Spack can currently do this for you as well.
+ After installing ``environment-modules`` following the step
+ above, source Spack's shell integration script. This will automatically
+ detect the lack of ``modulecmd`` and ``module``, and use the installed
+ ``environment-modules`` from ``spack bootstrap`` or ``spack install``.
+
+ .. code-block:: console
- $ spack install environment-modules
+ # For bash/zsh users
+ $ export SPACK_ROOT=/path/to/spack
+ $ . $SPACK_ROOT/share/spack/setup-env.sh
-#. Activate with the following script (or apply the updates to your
- ``.bashrc`` file manually):
- .. code-block:: sh
+ * If you prefer to do it manually, you can activate with the following
+ script (or apply the updates to your ``.bashrc`` file manually):
- TMP=`tempfile`
- echo >$TMP
- MODULE_HOME=`spack location --install-dir environment-modules`
- MODULE_VERSION=`ls -1 $MODULE_HOME/Modules | head -1`
- ${MODULE_HOME}/Modules/${MODULE_VERSION}/bin/add.modules <$TMP
- cp .bashrc $TMP
- echo "MODULE_VERSION=${MODULE_VERSION}" > .bashrc
- cat $TMP >>.bashrc
+ .. code-block:: sh
-This adds to your ``.bashrc`` (or similar) files, enabling Environment
-Modules when you log in. Re-load your .bashrc (or log out and in
-again), and then test that the ``module`` command is found with:
+ TMP=`tempfile`
+ echo >$TMP
+ MODULE_HOME=`spack location --install-dir environment-modules`
+ MODULE_VERSION=`ls -1 $MODULE_HOME/Modules | head -1`
+ ${MODULE_HOME}/Modules/${MODULE_VERSION}/bin/add.modules <$TMP
+ cp .bashrc $TMP
+ echo "MODULE_VERSION=${MODULE_VERSION}" > .bashrc
+ cat $TMP >>.bashrc
-.. code-block:: console
+ This is added to your ``.bashrc`` (or similar) files, enabling Environment
+ Modules when you log in.
+
+#. Test that the ``module`` command is found with:
- $ module avail
+ .. code-block:: console
+
+ $ module avail
+
+
+If ``tcl`` 8.0 or later is installed on your system, you can prevent
+spack from rebuilding ``tcl`` as part of the ``environment-modules`` dependency
+stack by adding the following to your ``~/.spack/packages.yaml`` replacing
+version 8.5 with whatever version is installed on your system:
+
+ .. code-block:: yaml
+ packages:
+ tcl:
+ paths:
+ tcl@8.5: /usr
+ buildable: False
^^^^^^^^^^^^^^^^^
Package Utilities
diff --git a/lib/spack/docs/module_file_support.rst b/lib/spack/docs/module_file_support.rst
index 93c2ee33c6..d8f5e561a8 100644
--- a/lib/spack/docs/module_file_support.rst
+++ b/lib/spack/docs/module_file_support.rst
@@ -44,6 +44,7 @@ For ``csh`` and ``tcsh`` instead:
$ source $SPACK_ROOT/share/spack/setup-env.csh
+When ``bash`` and ``ksh`` users update their environment with ``setup-env.sh``, it will check for spack-installed environment modules and add the ``module`` command to their environment; This only occurs if the module command is not already available. You can install ``environment-modules`` with ``spack bootstrap`` as described in :ref:`InstallEnvironmentModules`.
.. note::
You can put the source line in your ``.bashrc`` or ``.cshrc`` to
@@ -54,10 +55,11 @@ For ``csh`` and ``tcsh`` instead:
Using module files via Spack
----------------------------
-If you have shell support enabled you should be able to run either
-``module avail`` or ``use -l spack`` to see what module/dotkit files have
-been installed. Here is sample output of those programs, showing lots
-of installed packages.
+If you have installed a supported module system either manually or through
+``spack bootstrap`` and have enabled shell support, you should be able to
+run either ``module avail`` or ``use -l spack`` to see what module/dotkit
+files have been installed. Here is sample output of those programs,
+showing lots of installed packages.
.. code-block:: console
@@ -210,7 +212,7 @@ Scripts to load modules recursively may be made with the command:
$ spack module loads --dependencies <spec>
-An equivalent alternative is:
+An equivalent alternative using `process substitution <http://tldp.org/LDP/abs/html/process-sub.html>`_ is:
.. code-block :: console
diff --git a/lib/spack/spack/cmd/bootstrap.py b/lib/spack/spack/cmd/bootstrap.py
index b0550644f8..cd0a21b9e0 100644
--- a/lib/spack/spack/cmd/bootstrap.py
+++ b/lib/spack/spack/cmd/bootstrap.py
@@ -22,86 +22,69 @@
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
-import os
-
import llnl.util.tty as tty
-from llnl.util.filesystem import join_path, mkdirp
-
import spack
-from spack.util.executable import ProcessError, which
-
-_SPACK_UPSTREAM = 'https://github.com/llnl/spack'
+import spack.cmd
+import spack.cmd.common.arguments as arguments
-description = "create a new installation of spack in another prefix"
+description = "Bootstrap packages needed for spack to run smoothly"
section = "admin"
level = "long"
def setup_parser(subparser):
subparser.add_argument(
- '-r', '--remote', action='store', dest='remote',
- help="name of the remote to bootstrap from", default='origin')
+ '-j', '--jobs', action='store', type=int,
+ help="explicitly set number of make jobs. default is #cpus")
subparser.add_argument(
- 'prefix',
- help="names of prefix where we should install spack")
-
-
-def get_origin_info(remote):
- git_dir = join_path(spack.prefix, '.git')
- git = which('git', required=True)
- try:
- branch = git('symbolic-ref', '--short', 'HEAD', output=str)
- except ProcessError:
- branch = 'develop'
- tty.warn('No branch found; using default branch: %s' % branch)
- if remote == 'origin' and \
- branch not in ('master', 'develop'):
- branch = 'develop'
- tty.warn('Unknown branch found; using default branch: %s' % branch)
- try:
- origin_url = git(
- '--git-dir=%s' % git_dir,
- 'config', '--get', 'remote.%s.url' % remote,
- output=str)
- except ProcessError:
- origin_url = _SPACK_UPSTREAM
- tty.warn('No git repository found; '
- 'using default upstream URL: %s' % origin_url)
- return (origin_url.strip(), branch.strip())
-
-
-def bootstrap(parser, args):
- origin_url, branch = get_origin_info(args.remote)
- prefix = args.prefix
-
- tty.msg("Fetching spack from '%s': %s" % (args.remote, origin_url))
-
- if os.path.isfile(prefix):
- tty.die("There is already a file at %s" % prefix)
-
- mkdirp(prefix)
-
- if os.path.exists(join_path(prefix, '.git')):
- tty.die("There already seems to be a git repository in %s" % prefix)
-
- files_in_the_way = os.listdir(prefix)
- if files_in_the_way:
- tty.die("There are already files there! "
- "Delete these files before boostrapping spack.",
- *files_in_the_way)
-
- tty.msg("Installing:",
- "%s/bin/spack" % prefix,
- "%s/lib/spack/..." % prefix)
+ '--keep-prefix', action='store_true', dest='keep_prefix',
+ help="don't remove the install prefix if installation fails")
+ subparser.add_argument(
+ '--keep-stage', action='store_true', dest='keep_stage',
+ help="don't remove the build stage if installation succeeds")
+ subparser.add_argument(
+ '-n', '--no-checksum', action='store_true', dest='no_checksum',
+ help="do not check packages against checksum")
+ subparser.add_argument(
+ '-v', '--verbose', action='store_true', dest='verbose',
+ help="display verbose build output while installing")
- os.chdir(prefix)
- git = which('git', required=True)
- git('init', '--shared', '-q')
- git('remote', 'add', 'origin', origin_url)
- git('fetch', 'origin', '%s:refs/remotes/origin/%s' % (branch, branch),
- '-n', '-q')
- git('reset', '--hard', 'origin/%s' % branch, '-q')
- git('checkout', '-B', branch, 'origin/%s' % branch, '-q')
+ cd_group = subparser.add_mutually_exclusive_group()
+ arguments.add_common_arguments(cd_group, ['clean', 'dirty'])
- tty.msg("Successfully created a new spack in %s" % prefix,
- "Run %s/bin/spack to use this installation." % prefix)
+ subparser.add_argument(
+ '--run-tests', action='store_true', dest='run_tests',
+ help="run package level tests during installation"
+ )
+
+
+def bootstrap(parser, args, **kwargs):
+ kwargs.update({
+ 'keep_prefix': args.keep_prefix,
+ 'keep_stage': args.keep_stage,
+ 'install_deps': 'dependencies',
+ 'make_jobs': args.jobs,
+ 'run_tests': args.run_tests,
+ 'verbose': args.verbose,
+ 'dirty': args.dirty
+ })
+
+ # Define requirement dictionary defining general specs which need
+ # to be satisfied, and the specs to install when the general spec
+ # isn't satisfied.
+ requirement_dict = {'environment-modules': 'environment-modules~X'}
+
+ for requirement in requirement_dict:
+ installed_specs = spack.store.db.query(requirement)
+ if(len(installed_specs) > 0):
+ tty.msg("Requirement %s is satisfied with installed "
+ "package %s" % (requirement, installed_specs[0]))
+ else:
+ # Install requirement
+ spec_to_install = spack.Spec(requirement_dict[requirement])
+ spec_to_install.concretize()
+ tty.msg("Installing %s to satisfy requirement for %s" %
+ (spec_to_install, requirement))
+ kwargs['explicit'] = True
+ package = spack.repo.get(spec_to_install)
+ package.do_install(**kwargs)
diff --git a/lib/spack/spack/cmd/clone.py b/lib/spack/spack/cmd/clone.py
new file mode 100644
index 0000000000..19201b1f86
--- /dev/null
+++ b/lib/spack/spack/cmd/clone.py
@@ -0,0 +1,107 @@
+##############################################################################
+# Copyright (c) 2013-2016, Lawrence Livermore National Security, LLC.
+# Produced at the Lawrence Livermore National Laboratory.
+#
+# This file is part of Spack.
+# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved.
+# LLNL-CODE-647188
+#
+# For details, see https://github.com/llnl/spack
+# Please also see the LICENSE file for our notice and the LGPL.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License (as
+# published by the Free Software Foundation) version 2.1, February 1999.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and
+# conditions of the GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+##############################################################################
+import os
+
+import llnl.util.tty as tty
+from llnl.util.filesystem import join_path, mkdirp
+
+import spack
+from spack.util.executable import ProcessError, which
+
+_SPACK_UPSTREAM = 'https://github.com/llnl/spack'
+
+description = "create a new installation of spack in another prefix"
+section = "admin"
+level = "long"
+
+
+def setup_parser(subparser):
+ subparser.add_argument(
+ '-r', '--remote', action='store', dest='remote',
+ help="name of the remote to clone from", default='origin')
+ subparser.add_argument(
+ 'prefix',
+ help="names of prefix where we should install spack")
+
+
+def get_origin_info(remote):
+ git_dir = join_path(spack.prefix, '.git')
+ git = which('git', required=True)
+ try:
+ branch = git('symbolic-ref', '--short', 'HEAD', output=str)
+ except ProcessError:
+ branch = 'develop'
+ tty.warn('No branch found; using default branch: %s' % branch)
+ if remote == 'origin' and \
+ branch not in ('master', 'develop'):
+ branch = 'develop'
+ tty.warn('Unknown branch found; using default branch: %s' % branch)
+ try:
+ origin_url = git(
+ '--git-dir=%s' % git_dir,
+ 'config', '--get', 'remote.%s.url' % remote,
+ output=str)
+ except ProcessError:
+ origin_url = _SPACK_UPSTREAM
+ tty.warn('No git repository found; '
+ 'using default upstream URL: %s' % origin_url)
+ return (origin_url.strip(), branch.strip())
+
+
+def clone(parser, args):
+ origin_url, branch = get_origin_info(args.remote)
+ prefix = args.prefix
+
+ tty.msg("Fetching spack from '%s': %s" % (args.remote, origin_url))
+
+ if os.path.isfile(prefix):
+ tty.die("There is already a file at %s" % prefix)
+
+ mkdirp(prefix)
+
+ if os.path.exists(join_path(prefix, '.git')):
+ tty.die("There already seems to be a git repository in %s" % prefix)
+
+ files_in_the_way = os.listdir(prefix)
+ if files_in_the_way:
+ tty.die("There are already files there! "
+ "Delete these files before boostrapping spack.",
+ *files_in_the_way)
+
+ tty.msg("Installing:",
+ "%s/bin/spack" % prefix,
+ "%s/lib/spack/..." % prefix)
+
+ os.chdir(prefix)
+ git = which('git', required=True)
+ git('init', '--shared', '-q')
+ git('remote', 'add', 'origin', origin_url)
+ git('fetch', 'origin', '%s:refs/remotes/origin/%s' % (branch, branch),
+ '-n', '-q')
+ git('reset', '--hard', 'origin/%s' % branch, '-q')
+ git('checkout', '-B', branch, 'origin/%s' % branch, '-q')
+
+ tty.msg("Successfully created a new spack in %s" % prefix,
+ "Run %s/bin/spack to use this installation." % prefix)
diff --git a/share/spack/setup-env.sh b/share/spack/setup-env.sh
index c9a3858621..958689cab4 100755
--- a/share/spack/setup-env.sh
+++ b/share/spack/setup-env.sh
@@ -27,7 +27,9 @@
#
# This file is part of Spack and sets up the spack environment for
# bash and zsh. This includes dotkit support, module support, and
-# it also puts spack in your path. Source it like this:
+# it also puts spack in your path. The script also checks that
+# at least module support exists, and provides suggestions if it
+# doesn't. Source it like this:
#
# . /path/to/spack/share/spack/setup-env.sh
#
@@ -189,19 +191,68 @@ if [ -z "$_sp_source_file" ]; then
fi
#
-# Set up modules and dotkit search paths in the user environment
+# Find root directory and add bin to path.
#
_sp_share_dir=$(cd "$(dirname $_sp_source_file)" && pwd)
_sp_prefix=$(cd "$(dirname $(dirname $_sp_share_dir))" && pwd)
_spack_pathadd PATH "${_sp_prefix%/}/bin"
+export SPACK_ROOT=${_sp_prefix}
+
+#
+# Determine which shell is being used
+#
+function _spack_determine_shell() {
+ ps -p $$ | tail -n 1 | awk '{print $4}' | xargs basename
+}
+export SPACK_SHELL=$(_spack_determine_shell)
+
+#
+# Check whether a shell function of the given name is defined
+#
+function _spack_fn_exists() {
+ type $1 2>&1 | grep -q 'shell function'
+}
+need_module="no"
+if [ ! $(_spack_fn_exists use) ] && [ ! $(_spack_fn_exists module) ]; then
+ need_module="yes"
+fi;
+
+#
+# build and make available environment-modules
+#
+if [ "${need_module}" = "yes" ]; then
+ #check if environment-modules~X is installed
+ module_prefix="$(spack location -i "environment-modules" 2>&1 || echo "not_installed")"
+ module_prefix=$(echo ${module_prefix} | tail -n 1)
+ if [ "${module_prefix}" != "not_installed" ]; then
+ #activate it!
+ export MODULE_PREFIX=${module_prefix}
+ _spack_pathadd PATH "${MODULE_PREFIX}/Modules/bin"
+ module() { eval `${MODULE_PREFIX}/Modules/bin/modulecmd ${SPACK_SHELL} $*`; }
+ echo "INFO: Using spack managed module system."
+ else
+ echo "WARNING: A method for managing modules does not currently exist."
+ echo ""
+ echo "To resolve this you may either:"
+ echo "1. Allow spack to handle this by running 'spack boostrap'"
+ echo " and sourcing this script again."
+ echo "2. Install and activate a supported module managment engine manually"
+ echo " Supported engines include: environment-modules and lmod"
+ fi;
+else
+ echo "INFO: Using system available module system."
+fi;
+
+#
+# Set up modules and dotkit search paths in the user environment
+#
_sp_sys_type=$(spack-python -c 'print(spack.architecture.sys_type())')
_sp_dotkit_root=$(spack-python -c "print(spack.util.path.canonicalize_path(spack.config.get_config('config').get('module_roots', {}).get('dotkit')))")
_sp_tcl_root=$(spack-python -c "print(spack.util.path.canonicalize_path(spack.config.get_config('config').get('module_roots', {}).get('tcl')))")
_spack_pathadd DK_NODE "${_sp_dotkit_root%/}/$_sp_sys_type"
_spack_pathadd MODULEPATH "${_sp_tcl_root%/}/$_sp_sys_type"
-#
# Add programmable tab completion for Bash
#
if [ -n "${BASH_VERSION:-}" ]; then
diff --git a/var/spack/repos/builtin/packages/environment-modules/package.py b/var/spack/repos/builtin/packages/environment-modules/package.py
index 2774402605..fa0ea90fcf 100644
--- a/var/spack/repos/builtin/packages/environment-modules/package.py
+++ b/var/spack/repos/builtin/packages/environment-modules/package.py
@@ -35,6 +35,8 @@ class EnvironmentModules(Package):
version('3.2.10', '8b097fdcb90c514d7540bb55a3cb90fb')
+ variant('X', default=True, description='Build with X functionality')
+
# Dependencies:
depends_on('tcl', type=('build', 'link', 'run'))
@@ -75,6 +77,9 @@ class EnvironmentModules(Package):
'CPPFLAGS=' + ' '.join(cpp_flags)
]
+ if '~X' in spec:
+ config_args = ['--without-x'] + config_args
+
configure(*config_args)
make()
make('install')