summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2018-07-17Add documentation on build systems (#5015)Adam J. Stewart17-0/+3941
Spack provides a number of classes based on commonly-used build systems that users can extend when writing packages; the classes provide functionality to perform the actions relevant to the build system (e.g. running "configure" for an Autotools-based package). This adds documentation for classes supporting the following build systems: * Makefile * Autotools * CMake * QMake * SCons * Waf This includes build systems for managing extensions of the following packages: * Perl * Python * R * Octave This also adds documentation on implementing packages that use a custom build system (e.g. Perl/CMake). Spack also provides extendable classes which aggregate functionality for related sets of packages, e.g. those using CUDA. Documentation is added for CudaPackage.
2018-07-16add `spack --print-shell-vars` to speed up `setup-env.[c]sh` (#8101)Todd Gamblin2-0/+116
- The setup-env.sh script currently makes two calls to spack, but it should only need to make one. - Add a fast-path shell setup routine in `main.py` to allow the shell setup to happen in a single, fast call that doesn't load more than it needs to. - This simplifies setup code, as it has to eval what Spack prints - TODO: consider eventually making the whole setup script the output of a spack command
2018-07-14clean up of spack clean (#8610)Andreas Baumbach1-13/+15
* update help of `clean --all` to include `-p` * remove old orphaned `.pyc` removal * restrict removal or orphaned pyc files to `lib/spack` and `var/spack`
2018-07-12locks: improve errors and permission checkingTodd Gamblin2-52/+145
- Clean up error messages for when a lock can't be created, or when an exclusive (write) lock can't be taken on a file. - Add a number of subclasses of LockError to distinguish timeouts from permission issues. - Add an explicit check to prevent the user from taking a write lock on a read-only file. - We had a check for this for when we try to *upgrade* a lock on an RO file, but not for an initial write lock attempt. - Add more tests for different lock permission scenarios.
2018-07-12locks: llnl.util.lock now only writes host info when in debug modeTodd Gamblin2-13/+98
- write locks previously wrote information about the lock holder (host and pid), and read locks woudl read this in. - This is really only for debugging, so only enable it then - add some tests that target debug info, and improve multiproc lock test output
2018-07-12tests: Separate tests for llnl.util.lock and spack.util.lockTodd Gamblin4-161/+169
- llnl.util.lock tests are now independent of Spack
2018-07-12refactor: move spack.util.multiproc to llnl.util.multiprocTodd Gamblin5-12/+12
- multiproc doesn't depend on Spack - llnl.util.lock test uses it, but shouldn't use parts of Spack.
2018-07-11Don't propagate version-specific URL overrides to newer versions (#8565)Massimiliano Culpo2-18/+18
When a user specifies a URL for a specific version of a package, Spack originally would use that URL for all newer versions of the package. This behavior has proven to be generally more harmful than useful, so this PR removes the feature such that a version-specific URL override affects only that version.
2018-07-10fetch: remove 'trying' when cloning repos (#8651)Matthias Diener2-5/+5
* fetch: remove 'trying' when cloning repos This makes it similar to what the URL strategy prints. * change svn/hg names
2018-07-09Add ccache support (#3761)Christoph Junghans4-1/+37
If the user sets "ccache: true" in spack's config.yaml, Spack will use an available ccache executable when compiling c/c++ code. This feature is disabled by default (i.e. "ccache: false") and the documentation is updated with how to enable ccache support
2018-07-08fix typo in help of `spack clone` prefix is singular (#8658)Andreas Baumbach1-1/+1
Change-Id: I3bfe5b4ba497fae57d5382502ea9a1b054688fdb
2018-07-06Clear up a bit of modules/dotkit confusion (#8650)George Hartzell1-5/+7
`use` is an overloaded word between dotkit, modules and spack. Add additional words to make the distinction clear in the docs.
2018-06-29Removed duplicate word (#8609)Paul Chelarescu1-1/+1
2018-06-26views: packages can customize how they're added to views (#7152)scheibelp21-307/+794
Functional updates: - `python` now creates a copy of the `python` binaries when it is added to a view - Python extensions (packages which subclass `PythonPackage`) rewrite their shebang lines to refer to python in the view - Python packages in the same namespace will not generate conflicts if both have `...lib/site-packages/namespace-example/__init__.py` - These `__init__` files will also remain when removing any package in the namespace until the last package in the namespace is removed Generally (Updated 2/16): - Any package can define `add_files_to_view` to customize how it is added to a view (and at the moment custom definitions are included for `python` and `PythonPackage`) - Likewise any package can define `remove_files_from_view` to customize which files are removed (e.g. you don't always want to remove the namespace `__init__`) - Any package can define `view_file_conflicts` to customize what it considers a merge conflict - Global activations are handled like views (where the view root is the spec prefix of the extendee) - Benefit: filesystem-management aspects of activating extensions are now placed in views (e.g. now one can hardlink a global activation) - Benefit: overriding `Package.activate` is more straightforward (see `Python.activate`) - Complication: extension packages which have special-purpose logic *only* when activated outside of the extendee prefix must check for this in their `add_files_to_view` method (see `PythonPackage`) - `LinkTree` is refactored to have separate methods for copying a directory structure and for copying files (since it was found that generally packages may want to alter how files are copied but still wanted to copy directories in the same way) TODOs (updated 2/20): - [x] additional testing (there is some unit testing added at this point but more would be useful) - [x] refactor or reorganize `LinkTree` methods: currently there is a separate set of methods for replicating just the directory structure without the files, and a set for replicating everything - [x] Right now external views (i.e. those not used for global activations) call `view.add_extension`, but global activations do not to avoid some extra work that goes into maintaining external views. I'm not sure if addressing that needs to be done here but I'd like to clarify it in the comments (UPDATE: for now I have added a TODO and in my opinion this can be merged now and the refactor handled later) - [x] Several method descriptions (e.g. for `Package.activate`) are out of date and reference a distinction between global activations and views, they need to be updated - [x] Update aspell package activations
2018-06-26bugfix: fix macos incompatibility in lock test (#8573)Todd Gamblin3-16/+29
- Spack was assuming that a group with gid == current uid would always exist. - This was breaking the travis build for macos. - also fix issue with the DB tarball test finding coverage filesx
2018-06-25refactor: move pytest.ini and top-level conftest.py to lib/spack/spack/testTodd Gamblin3-1/+36
- removes two files from root of repository - `spack test` still works fine to run tests
2018-06-24refactor: move templates from root to share/spackTodd Gamblin3-15/+16
- This complies with the unix directory hierarchy standard (which Spack attempts to follow) - Also unclutters the repo root directory.
2018-06-24tests: test file/line attribution in config errorsTodd Gamblin3-30/+140
2018-06-24tests: add a test for `spack debug` commandTodd Gamblin3-2/+66
2018-06-24tests: test html output for `spack list`Todd Gamblin1-62/+37
- make list test use SpackCommand - convert to pytest - add a test for HTML output
2018-06-24Add `spack -H` shortcut for `spack help --all`Todd Gamblin1-18/+17
- I keep typing this based on `spack test -h` and `spack test -H` - This finally makes my fingers happy
2018-06-22Fix version detection for gcc@8 (#8511)Michael Kuhn1-2/+2
2018-06-20pytest: add _pytest/_version.py and LICENSETodd Gamblin4-1/+49
- pytest was not reporing the correct version from pytest.__version__. It reported 'unknown' - this fixes issues on some systems where system-installed pytest plugins would try to use the version and convert it to an int
2018-06-20externals: move spack.util.ordereddict to external/ordereddict_backportTodd Gamblin5-8/+5
2018-06-18Allow docs to be built with Sphinx 1.7.1+ (#8500)Adam J. Stewart1-1/+7
* Allow docs to be built with Sphinx 1.7.1+ * Allow docs to be built with Sphinx 2.0+
2018-06-17There's only one 'u' in "configuration" (#8498)George Hartzell1-1/+1
2018-06-17docs: remove vendored sphinx_rtd_theme (#8496)Todd Gamblin29-1124/+4
2018-06-14Update the clang compiler definition to use IBM XL Fortran compiler (#8389)Dan FitzGerald1-2/+6
on CORAL systems (linux-rhel7-ppc64le).
2018-06-12Add trailing slash when spidering URLs for versions (#8429)Adam J. Stewart1-0/+7
By default, if a package does not specify a list_url and does not download from a common repository, Spack runs dirname on the package URL. Given a URL like https://root.cern.ch/download/root_v6.09.02.source.tar.gz, this returns https://root.cern.ch/download. However, https://root.cern.ch/download gives a 404, while https://root.cern.ch/download/ works just fine. Note that some servers *don't* work with a trailing slash, so this tries with and without the slash. This will double the number of URLs searched but the slowdown should only affect the "spack versions" command.
2018-06-08compiler flags: add cxx98 standard support (#7601)Chris Green8-50/+285
The following improvements are made to cxx standard support (e.g. compiler.cxxNN_flag functions) in compilers: * Add cxx98_flag property * Add support for throwing an exception when a flag is not supported (previously if a flag was not supported the application was terminated with tty.die) * The name of the flag associated with e.g. c++14 standard support changes for different compiler versions (e.g. c++1y vs c++14). This makes a few corrections on what flag to return for which version. * Added tests to confirm that versions report expected flags for various c++ standards (or raise an exception for versions that don't provide a given cxx standard) Note that if a given cxx standard is the default, the associated flag property will return ""; cxx98 is assumed to be the default standard so this is the behavior for the associated property in the base compiler class. Package changes: * Improvements to the boost spec to take advantage of the improved standard flag facility. * Update the clingo spec to catch the new exception rather than look for an empty flag to indicate non-support (which is not part of the compiler flag API)
2018-06-07add python cache removal to `spack clean` (#8419)healther1-4/+24
Remove .pyc and .pyo files along with __pycache__directory if the user provides the -p/--python-cache option to "spack clean"
2018-06-06Fix bug where patches specified by dependents were not applied (#8272)scheibelp2-12/+13
Fixes #7885 #7193 added the patches_to_apply function to collect patches which are then applied in Package.do_patch. However this only collects patches that are associated with the Package object and does not include Spec-related patches (which are applied by dependents, added in #5476). Spec.patches already collects patches from the package as well as those applied by dependents, so the Package.patches_to_apply function isn't necessary. All uses of Package.patches_to_apply are replaced with Package.spec.patches. This also updates Package.content_hash to require the associated spec to be concrete: Spec.patches is only set after concretization. Before this PR, it was possible for Package.content_hash to be valid before concretizing the associated Spec if all patches were associated with the Package (vs. being applied by dependents). This behavior was unreliable though so the change is unlikely to be disruptive.
2018-06-05Preserve Spack CC/FC/F77/CXX settings when loading modules (#8346)Massimiliano Culpo4-15/+110
Fixes #8345 Spack environment modifications are applied before modules are loaded; this includes settings to CC, FC, F77, and CXX, which point to the Spack compiler wrappers. If the loaded modules set CC, this overrides the Spack compiler wrappers. This PR adds a context manager to preserve the values of CC etc. that are set by Spack: any effects on the CC, FC, F77, and CXX variables from modules are undone and their original values are restored.
2018-06-04pybind11: test functionality (#8304)Axel Huebl1-0/+2
* pybind11: test support Add a test functionality to pybind11. * CMake: test also on "make check" Some projects use non-CTest manual targets for tests.
2018-06-02Less sensitive error detection in build logs (#8278)Adam J. Stewart1-5/+2
* Less sensitive error detection in build logs * Fix test_log_parser unit test
2018-06-02Don't duplicate Python error messages (#8270)Adam J. Stewart1-1/+1
2018-06-02Fix coloring of error messages containing '}' symbol (#8277)Adam J. Stewart3-11/+30
2018-06-02Fix package error message line numbers (#8271)Adam J. Stewart1-12/+20
Line numbers were reported as zero-indexed, but we need to adjust.
2018-06-01Skip external specs when creating mirrors (#8084)Massimiliano Culpo2-5/+47
fixes #8083 External specs are supposed to be installed already, so there's no need to try to download a tarball for them.
2018-06-01Typo: an -> and (#8350)George Hartzell1-1/+1
2018-06-01extend Prefix class with join() member to support dynamic directories (#8329)Denis Davydov3-2/+44
* extend Prefix class with join() member to support dynamic directories * add more tests for Prefix.join() * more tests for Prefix.join() * add docstring * add example to docstring of Prefix class * cleanup Prefix.join() tests * use Prefix.join() in Packaging Guide
2018-05-31binary caching: skip check if old and new paths are identical (#8281)George Hartzell1-0/+2
Fixes #8217 Trying to relocate a distribution when the new and old paths are equal leads to failure, because the test that ensures that no unrelocated bits are left over always fails. As an example, this occurs if a user installs a package, generates a binary with it using 'spack buildcache', uninstalls it, and then attempts to reinstall into the same spack installation using the generated binary package. This updates the relocation check to accept the presence of the old prefix in binaries if the package is being reinstalled into its original location.
2018-05-31improve Advanced Compiler Configuration part of tutorial (#8325)Denis Davydov1-2/+21
2018-05-30more-flexible user-specified dependency constraints (#8162)scheibelp3-17/+80
* allow user to constrain dependencies that are added conditionally * remove check for not-visited deps from normalize, move it to concretize. The check now runs after the concretization loop completes (so an error is only reported if the user-mentioned spec doesnt appear anywhere in the dag) * remove separate full_spec_deps variable; rename spec_deps to all_spec_deps to clarify that it merges user-specified dependencies with derived dependencies * add unit test to confirm new functionality
2018-05-30config: add `spack config blame` command (#8081)Todd Gamblin5-10/+339
- `spack config blame` is similar to `spack config get`, but it prints out the config file and line number that each line of the merged configuration came from. - This is a debugging tool for understanding where Spack config settings come from. - add tests for config blame
2018-05-30Document specs with multiple version ranges (#8284)Adam J. Stewart1-0/+8
2018-05-24Fix missing spack.compiler import (#8240)Michael Kuhn1-0/+1
Fixes: #8258 #8090 altered import behavior so that import spack no longer provides access to many other Spack modules. This addresses a case which depended on the prior behavior and was not updated as part of #8090. This particular import error only came up when users were setting compiler flags on specs. See also: #8194
2018-05-22CUDA 9.2 (#8238)Axel Huebl1-4/+7
Add the latest CUDA release, v9.2 and updated CUDA Compatibility Matrix
2018-05-21Add info about haveged to gpg/randomness note (#8227)George Hartzell1-3/+18
2018-05-20URL parsing: strip x86-64 from filename (#8200)Adam J. Stewart2-2/+4