summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2019-05-27docs: set gettext_uuid to False to reduce churn (#11567)Todd Gamblin2-1/+2
- `gettext_uuid=True` makes every commit update every .pot file in spack/localized-docs, and speeds up the internationalized doc build slightly. - Optimize for less repository churn, and use `python-levenshtein` to accelerate the build instead.
2019-05-27mirror create: non-zero return code (#8585)Axel Huebl1-0/+2
`mirror create` should return a non-zero return code if errors occured.
2019-05-26link_tree: add option to merge link trees with relative targetsTodd Gamblin2-40/+90
- previous version of link trees would only do absolute symlinks - this version can do relative links using merge(relative=True)
2019-05-26docs: make docs build relocatable for localized buildsTodd Gamblin21-50/+59
- make all Spack paths relative to a `_spack_root` symlink, so that we can easily relocate the docs build *outside* lib/spack/docs - set some useful defaults for gettext translation variables in conf.py - update `relativeinclude` and other references to the spack root in the RST files to use _spack_root
2019-05-26docs: remove legacy import for Sphinx 1.7Todd Gamblin1-6/+1
2019-05-26docs: match ReadTheDocs; use -W (warnings as errors) with SphinxTodd Gamblin1-1/+1
2019-05-26commands: add --update option to `spack list`Todd Gamblin5-96/+171
- Add a `--update FILE` option to `spack list` - Output is written to the file only if any package is newer than the file - Simplify the code in docs/conf.py using this new option
2019-05-26commands: Add --header and --update options to `spack commands`Todd Gamblin4-54/+143
The Spack documentation currently hard-codes some functionality in `conf.py`, which makes the doc build less "pluggable" for things like localized doc builds. In particular, we unconditionally generate an index of commands and a package list as part of the docs, but those should really only be done if things are not up to date. This commit does the following: - Add `--header` option to `spack commands` so that it can do the work of prepending text to its output. - Add `--update FILE` option to `spack commands` that makes it generate a new command index *only* if FILE is out of date w.r.t. commands in the Spack source. - Simplify code in `conf.py` to use these options and only update the command index when needed.
2019-05-26docs: Use HDF5 as the example package for `spack graph`Todd Gamblin2-13/+17
2019-05-24build env: simplify handling of parallel jobs (#11524)Massimiliano Culpo9-35/+97
This PR implements several refactors requested in #11373, specifically: - Config scopes are used to handle builtin defaults, command line overrides and package overrides (`parallel=False`) - `Package.make_jobs` attribute has been removed; `make_jobs` remains as a module-scope variable in the build environment. - The use of the argument `-j` has been rationalized across commands - move '-j'/'--jobs' argument into `spack.cmd.common.arguments` - Add unit tests to check that setting parallel jobs works as expected - add new test to ensure that build job setting is isolated to each build - Fix packages that used `Package.make_jobs` (i.e. `bazel`)
2019-05-23docs: truncate `spack list` output in basic usageTodd Gamblin1-3/+6
`spack list` output is very long and takes up a lot of space in the docs. Truncate it to just 10 lines and link to the package list page.
2019-05-21Add Fujitsu compiler to Spack. (#11287)t-karatsu6-4/+86
* Add Fujitsu compiler to Spack. * Fixes for flake8 * Chenges location of FCC to subdirectory called case-insensitive * Add compiler tests for Fujitsu compiler * Modify the logic of taking compiler version for new version of Fujitsu compiler
2019-05-20Make flake8 check spack script as well as .py files (#11513)Adam J. Stewart1-1/+1
* Make flake8 check bin/spack
2019-05-20Get Cray OS version from PrgEnv-cray (#10774)Mario Melara1-2/+2
The regex used for finding the Cray OS version from the PrgEnv-cray module was not exact and was at times pulling the version from other PrgEnv modules. This updates the regular expression to be more exact.
2019-05-18Typos: funciton, woudl, hm,... (#11511)George Hartzell1-3/+3
2019-05-18Execute modulecmd in bash shell (#11510)Mario Melara1-2/+4
Adds executable=/bin/bash into Popen. We discovered this bug while working in a csh/tsch environment. By executing with /bin/bash we ensure that the module command works.
2019-05-16Command extensions can import code from modules in root or cmd folder (#11209)Massimiliano Culpo4-4/+277
#8612 added command extensions to Spack: a command implemented in a separate directory. This improves the implementation by allowing the command to import additional utility code stored within the established directory structure for commands. This also: * Adds tests for command extensions * Documents command extensions (including the expected directory layout)
2019-05-17Allow command access to dump/pickle_environment from #8476 (#11434)Chris Green2-7/+56
* Allow command access to dump/pickle_environment from #8476
2019-05-15fix suite-sparse built with tbb from intel-parallel-studio (#11134)Denis Davydov1-1/+11
* fix suite-sparse built with tbb from intel-parallel-studio * intel: add tbb_headers, strip newline from cxx_lib * use property
2019-05-15Use `svn info --xml` instead of `svn info` to get svn revisions (#11466)Todd Gamblin2-14/+8
- `svn info` prints different results depending on the system locale - in particular, Japanese output doesn't contain "Revision:" - Change Spack code to use XML output instead of using the human output
2019-05-14Remove vestigial print statement in module logic (#11438)Greg Becker1-1/+0
Remove a vestigial print statement introduced in #8570
2019-05-10permissions: preserve suid and sgid bits (#10727)Greg Becker2-0/+46
* Don't overwrite suid/sgid bits when setting permissions * add tests for permission setting
2019-05-10Intel packages: multiple installs and optional scalapack libs (#11384)Glenn Johnson1-1/+5
Add fixes to support multiple installs and dependents using a subset of IntelPackage functionality. * Update IntelPackage to only return scalapack libraries if the root spec depends on MPI: scalapack requires MPI to be mentioned as a dependency in the DAG. Package builds using intel-mkl for its blas/lapack implementations but not for scalapack were failing to build. Ideally it would be possible to ask if any of the packages in the DAG are actually requesting the scalapack functionality provided by the IntelPackage and only return scalapack libs in that case, but that is not easily done at this time. Fixes #11314 Fixes #11289 * set HOME when the intel silent installer is run. This prevents the installer from using the ~/intel directory (which can cause conflicts for multiple installs of the same IntelPackage) Fixes #9713
2019-05-09modules: use new `module` function instead of `get_module_cmd` (#8570)Greg Becker8-248/+118
Use new `module` function instead of `get_module_cmd` Previously, Spack relied on either examining the bash `module()` function or using the `which` command to find the underlying executable for modules. More complicated module systems do not allow for the sort of simple analysis we were doing (see #6451). Spack now uses the `module` function directly and copies environment changes from the resulting subprocess back into Spack. This should provide a future-proof implementation for changes to the logic underlying the module system on various HPC systems.
2019-05-10make license check more-strictPeter Josef Scheibel2-39/+93
The license text is now expected to match almost exactly (not accounting for formatting in different file types (e.g. rst vs. bash script vs. python)
2019-05-08Hide some variables in spack-build.env fileSeth R Johnson1-2/+6
Don't arbitrarily reset PWD and OLDPWD when sourcing, as well as other session-specific IDs
2019-05-07Environment path sanitization and sourcing (#8476)Chris Green3-7/+208
Add two functions to the EnvironmentModifications object to help users sanitize environment variables in their package definitions: * deprioritize_system_paths: this keeps system paths in the environment variable but moves them to the end. * prune_duplicate_paths: remove any duplicate paths from the variable This includes testing for the new functions as well as for (previously-untested) old convenience functions for environment variable manipulation. This also adds special handling for bash functions so they will be defined when the exported environment file is sourced.
2019-05-04Binary caches on MacOS - allow expanded RPATHs (#11345)Patrick Gartung2-0/+37
Fixes #11335 Update the Spack compiler wrappers to add the headerpad_max_install_names linker flag on MacOS. This allows the install_name_tool to rewrite the RPATH entry of the binary to be longer if needed. This is primarily useful for creating and distributing binary caches of packages (i.e. using the "spack buildcache" command); binary caches created on MacOS before this commit may not successfully relocate (if the target root path is larger).
2019-05-04Added a function that concretizes specs together (#11158)Massimiliano Culpo3-2/+112
* Added a function that concretizes specs together * Specs concretized together are copied instead of being referenced This makes the specs different objects and removes any reference to the fake root package that is needed currently for concretization. * Factored creating a repository for concretization into its own function * Added a test on overlapping dependencies
2019-05-03extend Version class so that 2.0 > 1.develop > 1.1 and develop > master > ↵Denis Davydov4-72/+123
head > trunk > 9999 (#1983) * extend Version class so that 2.0 > 1.develop > 1.1 * add concretization tests, with preferences and preferred version. * add master, head, trunk as develop-like versions, develop > master > head > trunk * update documentation on version comparison
2019-05-02fix error when adding package to env in container (#11321)Levi Baber1-1/+1
2019-05-02Don't use buildcache to install patchelf (#11343)Patrick Gartung1-1/+1
2019-05-01Fix spack flake8 to use Travis's target as base when neededTodd Gamblin1-2/+6
2019-04-24_valid_tokens list is missing intended comma (#11271)Daryl W. Grunau1-1/+1
* _valid_tokens list is missing a needed comma
2019-04-23Update tutorial page with RIKEN tutorial slides. (#11254)Todd Gamblin1-13/+28
- Add link to container image, as well as a description of VMs - Update slide link to point to latest RIKEN tutorial
2019-04-22Fix transitions between tutorial sections (#11251)Greg Becker3-11/+21
2019-04-20spack edit: use execv instead of Executable (#11245)Todd Gamblin3-37/+262
- `spack edit` previously used `spack.util.executable` `Executable` objects, and didn't `exec` the editor like you'd expect it to - This meant that Spack was still running while your editor was, and stdout/stdin were being set up in weird ways - e.g. on macOS, if you call `spack edit` with `EDITOR` set to the builtin `emacs` command, then type `Ctrl-g`, the whole thing dies with a `==> Error: Keyboard interrupt` - Fix all this by changing spack.util.editor to use `os.execv` instead of Spack's `Executable` object
2019-04-20Fix detection of LLVM-enabled PGI compilers (#10704)Adam J. Stewart2-2/+12
* Fix detection of LLVM-enabled PGI compilers * Add unit tests for LLVM-enabled PGI compiler version detection
2019-04-20Move NoLibrariesError/NoHeadersError into error.py (#10997)Denis Davydov3-11/+20
Also add constructor to NoLibrariesError which can either take an error message (like other SpackErrors) or a name and prefix (in which case the error message is constructed).
2019-04-19Fix outdated R packages failing to fetch (#11039)Justin S1-3/+4
PR #10758 made a slight change to find_versions_of_archive() which included archive_url in the search process. While this fixed `spack create` and `spack checksum` missing command-line arguments, it caused `spack install` to prefer those URLs over those it found in the scrape process. As a result, the package url was treated as a list_url causing all R packages to stop fetching once the package was updated on CRAN. This patch is more selective about including the archive_url in the remote versions, explicitly overriding it with matching versions found by the scraper.
2019-04-19Fix backwards compatibility for module naming (#11236)Greg Becker1-0/+4
f242f5f8 changed the format strings but maintained backwards compatibility in all cases except one: The list of valid tokens for the module naming schemes was not updated properly to contain both the new and old styles for compilers and package names. This PR re-adds the old tokens into the list of valid tokens.
2019-04-18Spack chain docs: config should go in upstreams.yaml (#11225)Peter Scheibel1-14/+12
#11152 added documentation for #8772 but some details were based on an earlier implementation that had changed by the time #8772 was merged. In particular, #11152 mentioned that upstream Spack instances were configured in config.yaml, when in fact they should be placed in a separate upstreams.yaml config file; this PR updates the documentation accordingly.
2019-04-18Moved cleanup before raising (the code couldn't be reached before)Massimiliano Culpo1-3/+3
2019-04-18Cleaned get methods of Repo and RepoPathMassimiliano Culpo1-5/+3
2019-04-18Used functools.wrap for the decorator + reordered importsMassimiliano Culpo1-20/+23
2019-04-18Removed 'namespace' argument from Repo and RepoPathMassimiliano Culpo1-32/+19
fixes #11159 The 'namespace' argument to both Repo and RepoPath were used to set the "super namespace". Currently it seems to be vestigial as the only "super namespace" allowed for packages is 'spack.pkg' since 39c9bbf
2019-04-18improvements to our CDash reporter (#11168)Zack Galbreath2-84/+127
* Make a separate CDash report for each package installed Previously, we generated a single CDash report ("build") for the complete results of running a `spack install` command. Now we create a separate CDash build for each package that was installed. This commit also changes some of the tests related to CDash reporting. Now only one of the tests exercises the code path of uploading to a (nonexistent) CDash server. The rest of the related tests write their reports to disk without trying to upload them. * Don't report errors to CDash for successful packages Convert errors detected by our log scraper into warnings when the package being installed reports that it was successful. * Report a maximum of 50 errors/warnings to CDash This is in line with what CTest does. The idea is that if you have more than 50 errors/warnings you probably aren't going to read through them all anyway. This change reduces the amount of data that we need to transfer and store.
2019-04-17Features: Improve Spec format strings (#10556)Greg Becker39-177/+561
* Update spec format to simpler syntax, maintain backwards compatibility * Switch to new spec.format method throughout internals * update package files for new format strings * documentation and minor code cleanup. removed nonsensical variant sigils
2019-04-17Add ftn to lib/spack/env (#11180)Satish Balay1-0/+1
Fixes #11070 #11010 Spack attempts to intercede on behalf of all compiler invocations for a build. This involves adding its wrappers to PATH. Cray systems include a "ftn" executable and Spack was only redirecting this call when the Spec was built with cce. This updates the compiler wrappers to add "ftn" in all cases.
2019-04-16don't record 'view: True' in environment config (#11182)Peter Scheibel1-1/+8
The default (implied) behavior for all environments, as of ea1de6b, is that an environment will maintain a view in a location of its choosing. ea1de6b explicitly recorded all three possible states of maintaining a view: 1. Maintain a view, and let the environment decide where to put it (default) 2. Maintain a view, and let the user decide 3. Don't maintain a view This commit updates the config writer so that for case [1], nothing will be written to the config.yaml. This will not change any existing behavior, it just serves to keep the config more compact.