summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2018-07-25tests: convert mirror test to pytest with free functions.Todd Gamblin1-44/+47
2018-07-25tests: add checks and tests for consistent version() argumentsTodd Gamblin3-2/+60
- Previously, Spack didn't check the arguments you put in version() directives. - So, you could do something like this, where there are arguments for a URL fetcher AND for a git fetcher: version('1.0', md5='abc123', git='https://foo.bar', commit='feda2343') - Now, we check the arguments before constructing a fetcher, to ensure that each package has *only* arguments for a single type of fetcher. - Also added `test_package_version_consistency()` to the `package_sanity` test, so that all builtin packages are required to have valid `version()` directives.
2018-07-25tests: correct name of `check_db` to `check_repo`Todd Gamblin1-7/+6
- this test is checking the package *repository*, not the database.
2018-07-25core: differentiate package-level fetch URLs by args to `version()`Todd Gamblin2-84/+186
- packagers can specify two top-level fetch URLs if one is `url` - e.g., `url` and `git` or `url` and `svn` - allow only one VCS fetcher so we can differentiate between URL and VCS. - also clean up fetcher logic and class structure
2018-07-25core: add check for conflicting top-level fetch attributes in packagesTodd Gamblin4-6/+35
- ensure that packages can't have more than one of git, hg, svn, or url
2018-07-25style: clean up exception definitions in package.pyTodd Gamblin1-7/+2
2018-07-25core: fixes and tests for handling of fetcher attributes in packagesTodd Gamblin3-40/+174
- Packages can remove the top-level `url` attribute and still work - These are now legal: - Packages with *only* version-specific URLs (even with gaps) - Packages with a top-level git/hg/svn attribute and `version` directives for that. - If a package has both a top-level hg/git/svn attribute AND a top-level url attribute, the url attribute takes precedence.
2018-07-25core: quick fix for packages without URLsAdam J. Stewart1-11/+0
Some packages do not have a `url` and are instead downloaded via `git`, `hg`, or `svn`. Some packages like `spectrum-mpi` cannot be downloaded at all, and are placeholder packages for system installations. Previously, `__init__()` in `PackageBase` crashed if a package did not have a `url` attribute defined. I hacked this section of code out, but I have no idea what the repercussions of that are.
2018-07-24Fix order of regexes reported by spack url summaryAdam J. Stewart1-2/+2
2018-07-24core: make `spack.util.crypto` initialization less expensive.Todd Gamblin2-38/+54
- This hard-codes the hash lengths rather than computing them on import. - Also cleans up the code in `spack.util.crypto` to make it easier to understand.
2018-07-24bugfix: `spack module -m` error message (#8794)Todd Gamblin1-3/+2
Fix this output error: ``` $ spack -m module loads mpileaks ==> Error: `spack module loads -m t -m c -m l ...` has been moved. Try this instead: $ spack module t loads mpileaks $ spack module c loads mpileaks $ spack module l loads mpileaks ```
2018-07-24Die with a meaningful error if a deprecated command is usedMassimiliano Culpo2-3/+57
In case a deprecated form of the module command is used, the program will exit non-zero and print an informative error message suggesting which command should be used instead.
2018-07-24Grouped all the module commands under `spack module`Massimiliano Culpo13-390/+266
As requested in the review all the commands meant to manage module files have been grouped under the `spack module` command. Unit tests have been refactored to match the new command structure.
2018-07-24module file tutorial: rebuilt docker image and updated the scriptMassimiliano Culpo1-170/+216
2018-07-24Added --dependencies to the help text of spack loadMassimiliano Culpo2-3/+5
fixes #6321
2018-07-24spack load exits with 1 if module does not exist or is not installedMassimiliano Culpo2-7/+68
fixes #2215 fixes #2570 fixes #6676 fixes #7281 closes #3827 This PR reverts the use of `spack module loads` in favor of `spack module find` when loading module files via Spack. After this PR `spack load` will accept a single spec at a time, and will be able to interpret correctly the `--dependencies` option.
2018-07-24Added blacklisting of implicit modules in docs + regression testsMassimiliano Culpo5-1/+52
fixes #4400 The feature requested in #4400 was already part of the module file configuration, but it was neither tested nor documented. This commit takes care of adding a few lines in the documentation and a regression test.
2018-07-24Renamed 'patch_configuration' fixture to 'module_configuration'Massimiliano Culpo5-78/+79
This just because the fixture has been moved one level above the one it was originally defined. In this more general context there's more than one configuration file that could be patched for tests.
2018-07-24Added a unit test for 'spack lmod setdefault'Massimiliano Culpo4-12/+101
2018-07-24Updated references to `spack module` in the documentation.Massimiliano Culpo5-55/+46
2018-07-24Added 'setdefault' subcommand to 'spack lmod'Massimiliano Culpo2-12/+38
Added a subcommand that sets the default module to be loaded, if multiple installations of the same package are present.
2018-07-24Split 'spack module' into multiple commandsMassimiliano Culpo10-194/+413
'spack module' has been split into multiple commands, each one tied to a specific module type. This permits the specialization of the new commands with features that are module type specific (e.g. set the default module file in lmod when multiple versions of the same package are installed at the same time).
2018-07-24bugfix: fix spack spec --yamlTodd Gamblin3-1/+58
- repo membership test was broken by the refactor of spack/__init__.py - refactor singleton so that 'spec in repo' works again for `spack.repo.path` - fix spec command and add basic tests for `spack spec` and `spack spec --yaml`
2018-07-23docs: update old docs for `spack.package.PackageBase`Todd Gamblin1-205/+37
- There was a lot of documentation in `PackageBase` dating back to the very first versions of Spack. - It was repetitive and out of date, and the docs at spack.readthedocs.io are better. - Remove the outdated specifics, and leave the minimal useful set of developer docs in `package.py`.
2018-07-23core: use sha256 instead of md5 for `spack checksum` and `spack create`Todd Gamblin5-36/+95
- This changes `get_checksums_for_versions` to generate code that uses an explicit `sha256` argument instead if the bare `md5` hash we used to generate. - also use a generic digest parameter for the `version` directive, rather than a specific `md5` parameter.
2018-07-21bugfix: stage test no longer modifies ~/.spack/config.yamlTodd Gamblin1-2/+2
- two stage tests weren't properly using the mutable_config fixture. - this caused running `spack test` to modify the user's config.yaml
2018-07-21docs: doucment `spack config blame`Todd Gamblin1-4/+59
2018-07-21docs: updates to `spack config get` docs for command-line scopesTodd Gamblin1-21/+73
2018-07-21docs: command-line scopesElizabeth Fischer1-3/+111
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2018-07-21config: allow user to add configuration scopes on the command line.Elizabeth Fischer4-72/+194
- Add command-line scope option to Spack - Rework structure of main to allow configuration system to raise errors more naturally Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2018-07-21locks: fix bug when creating lockfiles in the current directory.Todd Gamblin2-0/+25
- Fixes a bug in `llnl.util.lock` - Locks in the current directory would fail because the parent directory was the empty string. - Fix this and return '.' for the parent of locks in the current directory.
2018-07-19style: make core comply with pep8-namingTodd Gamblin30-94/+93
2018-07-19add py-pep8-naming packageTodd Gamblin1-5/+10
2018-07-18Better Makefile target detection (#8223)Adam J. Stewart1-21/+46
Replace regex-based target detection for Makefiles with a preliminary "make -q" to check if a target exists. This does not work for NetBSD make; additional work is required to detect if NetBSD make is present and to use a regex in that case. The affected makefile target checks are only performed when the "--test" flag is added to a "spack install" invocation.
2018-07-17shell: fixed a typo in spack --print-shell-vars (#8732)Massimiliano Culpo2-6/+6
2018-07-17Packages must be added to DB to be considered installed (#8038)Massimiliano Culpo3-6/+63
Fixes #8036 Before this PR Package.installed was returning True if the spec prefix existed, without checking the DB. This is wrong for external packages, whose prefix exists before being registered into the DB. Now the property checks for both the prefix and a DB entry.
2018-07-17Add documentation on build systems (#5015)Adam J. Stewart17-0/+3941
Spack provides a number of classes based on commonly-used build systems that users can extend when writing packages; the classes provide functionality to perform the actions relevant to the build system (e.g. running "configure" for an Autotools-based package). This adds documentation for classes supporting the following build systems: * Makefile * Autotools * CMake * QMake * SCons * Waf This includes build systems for managing extensions of the following packages: * Perl * Python * R * Octave This also adds documentation on implementing packages that use a custom build system (e.g. Perl/CMake). Spack also provides extendable classes which aggregate functionality for related sets of packages, e.g. those using CUDA. Documentation is added for CudaPackage.
2018-07-16add `spack --print-shell-vars` to speed up `setup-env.[c]sh` (#8101)Todd Gamblin2-0/+116
- The setup-env.sh script currently makes two calls to spack, but it should only need to make one. - Add a fast-path shell setup routine in `main.py` to allow the shell setup to happen in a single, fast call that doesn't load more than it needs to. - This simplifies setup code, as it has to eval what Spack prints - TODO: consider eventually making the whole setup script the output of a spack command
2018-07-14clean up of spack clean (#8610)Andreas Baumbach1-13/+15
* update help of `clean --all` to include `-p` * remove old orphaned `.pyc` removal * restrict removal or orphaned pyc files to `lib/spack` and `var/spack`
2018-07-12locks: improve errors and permission checkingTodd Gamblin2-52/+145
- Clean up error messages for when a lock can't be created, or when an exclusive (write) lock can't be taken on a file. - Add a number of subclasses of LockError to distinguish timeouts from permission issues. - Add an explicit check to prevent the user from taking a write lock on a read-only file. - We had a check for this for when we try to *upgrade* a lock on an RO file, but not for an initial write lock attempt. - Add more tests for different lock permission scenarios.
2018-07-12locks: llnl.util.lock now only writes host info when in debug modeTodd Gamblin2-13/+98
- write locks previously wrote information about the lock holder (host and pid), and read locks woudl read this in. - This is really only for debugging, so only enable it then - add some tests that target debug info, and improve multiproc lock test output
2018-07-12tests: Separate tests for llnl.util.lock and spack.util.lockTodd Gamblin4-161/+169
- llnl.util.lock tests are now independent of Spack
2018-07-12refactor: move spack.util.multiproc to llnl.util.multiprocTodd Gamblin5-12/+12
- multiproc doesn't depend on Spack - llnl.util.lock test uses it, but shouldn't use parts of Spack.
2018-07-11Don't propagate version-specific URL overrides to newer versions (#8565)Massimiliano Culpo2-18/+18
When a user specifies a URL for a specific version of a package, Spack originally would use that URL for all newer versions of the package. This behavior has proven to be generally more harmful than useful, so this PR removes the feature such that a version-specific URL override affects only that version.
2018-07-10fetch: remove 'trying' when cloning repos (#8651)Matthias Diener2-5/+5
* fetch: remove 'trying' when cloning repos This makes it similar to what the URL strategy prints. * change svn/hg names
2018-07-09Add ccache support (#3761)Christoph Junghans4-1/+37
If the user sets "ccache: true" in spack's config.yaml, Spack will use an available ccache executable when compiling c/c++ code. This feature is disabled by default (i.e. "ccache: false") and the documentation is updated with how to enable ccache support
2018-07-08fix typo in help of `spack clone` prefix is singular (#8658)Andreas Baumbach1-1/+1
Change-Id: I3bfe5b4ba497fae57d5382502ea9a1b054688fdb
2018-07-06Clear up a bit of modules/dotkit confusion (#8650)George Hartzell1-5/+7
`use` is an overloaded word between dotkit, modules and spack. Add additional words to make the distinction clear in the docs.
2018-06-29Removed duplicate word (#8609)Paul Chelarescu1-1/+1
2018-06-26views: packages can customize how they're added to views (#7152)scheibelp21-307/+794
Functional updates: - `python` now creates a copy of the `python` binaries when it is added to a view - Python extensions (packages which subclass `PythonPackage`) rewrite their shebang lines to refer to python in the view - Python packages in the same namespace will not generate conflicts if both have `...lib/site-packages/namespace-example/__init__.py` - These `__init__` files will also remain when removing any package in the namespace until the last package in the namespace is removed Generally (Updated 2/16): - Any package can define `add_files_to_view` to customize how it is added to a view (and at the moment custom definitions are included for `python` and `PythonPackage`) - Likewise any package can define `remove_files_from_view` to customize which files are removed (e.g. you don't always want to remove the namespace `__init__`) - Any package can define `view_file_conflicts` to customize what it considers a merge conflict - Global activations are handled like views (where the view root is the spec prefix of the extendee) - Benefit: filesystem-management aspects of activating extensions are now placed in views (e.g. now one can hardlink a global activation) - Benefit: overriding `Package.activate` is more straightforward (see `Python.activate`) - Complication: extension packages which have special-purpose logic *only* when activated outside of the extendee prefix must check for this in their `add_files_to_view` method (see `PythonPackage`) - `LinkTree` is refactored to have separate methods for copying a directory structure and for copying files (since it was found that generally packages may want to alter how files are copied but still wanted to copy directories in the same way) TODOs (updated 2/20): - [x] additional testing (there is some unit testing added at this point but more would be useful) - [x] refactor or reorganize `LinkTree` methods: currently there is a separate set of methods for replicating just the directory structure without the files, and a set for replicating everything - [x] Right now external views (i.e. those not used for global activations) call `view.add_extension`, but global activations do not to avoid some extra work that goes into maintaining external views. I'm not sure if addressing that needs to be done here but I'd like to clarify it in the comments (UPDATE: for now I have added a TODO and in my opinion this can be merged now and the refactor handled later) - [x] Several method descriptions (e.g. for `Package.activate`) are out of date and reference a distinction between global activations and views, they need to be updated - [x] Update aspell package activations