summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2019-06-04modules: set permissions based on package configuration (#11337)Greg Becker5-52/+99
Previously, module files were not set with the same permissions as the package installation. For world-readable packages, this would not cause a problem. For group readable packages, it does: ``` packages: mypackage: permissions: group: mygroup read: group write: group ``` In this case, the modulefile is unreadable by members of the group other than the one who installed it. Add logic to the modulefile writers to set the permissions based on the configuration in `packages.yaml`
2019-06-04Buildcache relocate.py error fix (#11616)Patrick Gartung1-0/+2
* Add a trailing / if file --mime does not return a mimetype with a / in it * Flake8
2019-05-31Build cache: relocate path to spack/bin/sbang in text files. (#11592)Patrick Gartung3-13/+35
* Build cache: relocate path to spack/bin/sbang in text files. * Found in testing. * update packaging test * Make sbang replacement including #!/bin/bash. Add an additional spack prefix replacement to fix stage directory references. * flake8 * Use buildinfo.get() so old buildcaches without buildinfo['spackprefix'] can be read.
2019-05-28Cap the maximum number of build jobs (#11373)Massimiliano Culpo5-16/+49
* config:build_jobs now controls the number of parallel jobs to spawn during builds, but cannot ever exceed the number of cores on the machine. * The default is set to 16 or the number of available cores, whatever is lowest. * Updated docs to reflect the changes done to limit parallel builds
2019-05-27docs: set gettext_uuid to False to reduce churn (#11567)Todd Gamblin2-1/+2
- `gettext_uuid=True` makes every commit update every .pot file in spack/localized-docs, and speeds up the internationalized doc build slightly. - Optimize for less repository churn, and use `python-levenshtein` to accelerate the build instead.
2019-05-27mirror create: non-zero return code (#8585)Axel Huebl1-0/+2
`mirror create` should return a non-zero return code if errors occured.
2019-05-26link_tree: add option to merge link trees with relative targetsTodd Gamblin2-40/+90
- previous version of link trees would only do absolute symlinks - this version can do relative links using merge(relative=True)
2019-05-26docs: make docs build relocatable for localized buildsTodd Gamblin21-50/+59
- make all Spack paths relative to a `_spack_root` symlink, so that we can easily relocate the docs build *outside* lib/spack/docs - set some useful defaults for gettext translation variables in conf.py - update `relativeinclude` and other references to the spack root in the RST files to use _spack_root
2019-05-26docs: remove legacy import for Sphinx 1.7Todd Gamblin1-6/+1
2019-05-26docs: match ReadTheDocs; use -W (warnings as errors) with SphinxTodd Gamblin1-1/+1
2019-05-26commands: add --update option to `spack list`Todd Gamblin5-96/+171
- Add a `--update FILE` option to `spack list` - Output is written to the file only if any package is newer than the file - Simplify the code in docs/conf.py using this new option
2019-05-26commands: Add --header and --update options to `spack commands`Todd Gamblin4-54/+143
The Spack documentation currently hard-codes some functionality in `conf.py`, which makes the doc build less "pluggable" for things like localized doc builds. In particular, we unconditionally generate an index of commands and a package list as part of the docs, but those should really only be done if things are not up to date. This commit does the following: - Add `--header` option to `spack commands` so that it can do the work of prepending text to its output. - Add `--update FILE` option to `spack commands` that makes it generate a new command index *only* if FILE is out of date w.r.t. commands in the Spack source. - Simplify code in `conf.py` to use these options and only update the command index when needed.
2019-05-26docs: Use HDF5 as the example package for `spack graph`Todd Gamblin2-13/+17
2019-05-24build env: simplify handling of parallel jobs (#11524)Massimiliano Culpo9-35/+97
This PR implements several refactors requested in #11373, specifically: - Config scopes are used to handle builtin defaults, command line overrides and package overrides (`parallel=False`) - `Package.make_jobs` attribute has been removed; `make_jobs` remains as a module-scope variable in the build environment. - The use of the argument `-j` has been rationalized across commands - move '-j'/'--jobs' argument into `spack.cmd.common.arguments` - Add unit tests to check that setting parallel jobs works as expected - add new test to ensure that build job setting is isolated to each build - Fix packages that used `Package.make_jobs` (i.e. `bazel`)
2019-05-23docs: truncate `spack list` output in basic usageTodd Gamblin1-3/+6
`spack list` output is very long and takes up a lot of space in the docs. Truncate it to just 10 lines and link to the package list page.
2019-05-21Add Fujitsu compiler to Spack. (#11287)t-karatsu6-4/+86
* Add Fujitsu compiler to Spack. * Fixes for flake8 * Chenges location of FCC to subdirectory called case-insensitive * Add compiler tests for Fujitsu compiler * Modify the logic of taking compiler version for new version of Fujitsu compiler
2019-05-20Make flake8 check spack script as well as .py files (#11513)Adam J. Stewart1-1/+1
* Make flake8 check bin/spack
2019-05-20Get Cray OS version from PrgEnv-cray (#10774)Mario Melara1-2/+2
The regex used for finding the Cray OS version from the PrgEnv-cray module was not exact and was at times pulling the version from other PrgEnv modules. This updates the regular expression to be more exact.
2019-05-18Typos: funciton, woudl, hm,... (#11511)George Hartzell1-3/+3
2019-05-18Execute modulecmd in bash shell (#11510)Mario Melara1-2/+4
Adds executable=/bin/bash into Popen. We discovered this bug while working in a csh/tsch environment. By executing with /bin/bash we ensure that the module command works.
2019-05-16Command extensions can import code from modules in root or cmd folder (#11209)Massimiliano Culpo4-4/+277
#8612 added command extensions to Spack: a command implemented in a separate directory. This improves the implementation by allowing the command to import additional utility code stored within the established directory structure for commands. This also: * Adds tests for command extensions * Documents command extensions (including the expected directory layout)
2019-05-17Allow command access to dump/pickle_environment from #8476 (#11434)Chris Green2-7/+56
* Allow command access to dump/pickle_environment from #8476
2019-05-15fix suite-sparse built with tbb from intel-parallel-studio (#11134)Denis Davydov1-1/+11
* fix suite-sparse built with tbb from intel-parallel-studio * intel: add tbb_headers, strip newline from cxx_lib * use property
2019-05-15Use `svn info --xml` instead of `svn info` to get svn revisions (#11466)Todd Gamblin2-14/+8
- `svn info` prints different results depending on the system locale - in particular, Japanese output doesn't contain "Revision:" - Change Spack code to use XML output instead of using the human output
2019-05-14Remove vestigial print statement in module logic (#11438)Greg Becker1-1/+0
Remove a vestigial print statement introduced in #8570
2019-05-10permissions: preserve suid and sgid bits (#10727)Greg Becker2-0/+46
* Don't overwrite suid/sgid bits when setting permissions * add tests for permission setting
2019-05-10Intel packages: multiple installs and optional scalapack libs (#11384)Glenn Johnson1-1/+5
Add fixes to support multiple installs and dependents using a subset of IntelPackage functionality. * Update IntelPackage to only return scalapack libraries if the root spec depends on MPI: scalapack requires MPI to be mentioned as a dependency in the DAG. Package builds using intel-mkl for its blas/lapack implementations but not for scalapack were failing to build. Ideally it would be possible to ask if any of the packages in the DAG are actually requesting the scalapack functionality provided by the IntelPackage and only return scalapack libs in that case, but that is not easily done at this time. Fixes #11314 Fixes #11289 * set HOME when the intel silent installer is run. This prevents the installer from using the ~/intel directory (which can cause conflicts for multiple installs of the same IntelPackage) Fixes #9713
2019-05-09modules: use new `module` function instead of `get_module_cmd` (#8570)Greg Becker8-248/+118
Use new `module` function instead of `get_module_cmd` Previously, Spack relied on either examining the bash `module()` function or using the `which` command to find the underlying executable for modules. More complicated module systems do not allow for the sort of simple analysis we were doing (see #6451). Spack now uses the `module` function directly and copies environment changes from the resulting subprocess back into Spack. This should provide a future-proof implementation for changes to the logic underlying the module system on various HPC systems.
2019-05-10make license check more-strictPeter Josef Scheibel2-39/+93
The license text is now expected to match almost exactly (not accounting for formatting in different file types (e.g. rst vs. bash script vs. python)
2019-05-08Hide some variables in spack-build.env fileSeth R Johnson1-2/+6
Don't arbitrarily reset PWD and OLDPWD when sourcing, as well as other session-specific IDs
2019-05-07Environment path sanitization and sourcing (#8476)Chris Green3-7/+208
Add two functions to the EnvironmentModifications object to help users sanitize environment variables in their package definitions: * deprioritize_system_paths: this keeps system paths in the environment variable but moves them to the end. * prune_duplicate_paths: remove any duplicate paths from the variable This includes testing for the new functions as well as for (previously-untested) old convenience functions for environment variable manipulation. This also adds special handling for bash functions so they will be defined when the exported environment file is sourced.
2019-05-04Binary caches on MacOS - allow expanded RPATHs (#11345)Patrick Gartung2-0/+37
Fixes #11335 Update the Spack compiler wrappers to add the headerpad_max_install_names linker flag on MacOS. This allows the install_name_tool to rewrite the RPATH entry of the binary to be longer if needed. This is primarily useful for creating and distributing binary caches of packages (i.e. using the "spack buildcache" command); binary caches created on MacOS before this commit may not successfully relocate (if the target root path is larger).
2019-05-04Added a function that concretizes specs together (#11158)Massimiliano Culpo3-2/+112
* Added a function that concretizes specs together * Specs concretized together are copied instead of being referenced This makes the specs different objects and removes any reference to the fake root package that is needed currently for concretization. * Factored creating a repository for concretization into its own function * Added a test on overlapping dependencies
2019-05-03extend Version class so that 2.0 > 1.develop > 1.1 and develop > master > ↵Denis Davydov4-72/+123
head > trunk > 9999 (#1983) * extend Version class so that 2.0 > 1.develop > 1.1 * add concretization tests, with preferences and preferred version. * add master, head, trunk as develop-like versions, develop > master > head > trunk * update documentation on version comparison
2019-05-02fix error when adding package to env in container (#11321)Levi Baber1-1/+1
2019-05-02Don't use buildcache to install patchelf (#11343)Patrick Gartung1-1/+1
2019-05-01Fix spack flake8 to use Travis's target as base when neededTodd Gamblin1-2/+6
2019-04-24_valid_tokens list is missing intended comma (#11271)Daryl W. Grunau1-1/+1
* _valid_tokens list is missing a needed comma
2019-04-23Update tutorial page with RIKEN tutorial slides. (#11254)Todd Gamblin1-13/+28
- Add link to container image, as well as a description of VMs - Update slide link to point to latest RIKEN tutorial
2019-04-22Fix transitions between tutorial sections (#11251)Greg Becker3-11/+21
2019-04-20spack edit: use execv instead of Executable (#11245)Todd Gamblin3-37/+262
- `spack edit` previously used `spack.util.executable` `Executable` objects, and didn't `exec` the editor like you'd expect it to - This meant that Spack was still running while your editor was, and stdout/stdin were being set up in weird ways - e.g. on macOS, if you call `spack edit` with `EDITOR` set to the builtin `emacs` command, then type `Ctrl-g`, the whole thing dies with a `==> Error: Keyboard interrupt` - Fix all this by changing spack.util.editor to use `os.execv` instead of Spack's `Executable` object
2019-04-20Fix detection of LLVM-enabled PGI compilers (#10704)Adam J. Stewart2-2/+12
* Fix detection of LLVM-enabled PGI compilers * Add unit tests for LLVM-enabled PGI compiler version detection
2019-04-20Move NoLibrariesError/NoHeadersError into error.py (#10997)Denis Davydov3-11/+20
Also add constructor to NoLibrariesError which can either take an error message (like other SpackErrors) or a name and prefix (in which case the error message is constructed).
2019-04-19Fix outdated R packages failing to fetch (#11039)Justin S1-3/+4
PR #10758 made a slight change to find_versions_of_archive() which included archive_url in the search process. While this fixed `spack create` and `spack checksum` missing command-line arguments, it caused `spack install` to prefer those URLs over those it found in the scrape process. As a result, the package url was treated as a list_url causing all R packages to stop fetching once the package was updated on CRAN. This patch is more selective about including the archive_url in the remote versions, explicitly overriding it with matching versions found by the scraper.
2019-04-19Fix backwards compatibility for module naming (#11236)Greg Becker1-0/+4
f242f5f8 changed the format strings but maintained backwards compatibility in all cases except one: The list of valid tokens for the module naming schemes was not updated properly to contain both the new and old styles for compilers and package names. This PR re-adds the old tokens into the list of valid tokens.
2019-04-18Spack chain docs: config should go in upstreams.yaml (#11225)Peter Scheibel1-14/+12
#11152 added documentation for #8772 but some details were based on an earlier implementation that had changed by the time #8772 was merged. In particular, #11152 mentioned that upstream Spack instances were configured in config.yaml, when in fact they should be placed in a separate upstreams.yaml config file; this PR updates the documentation accordingly.
2019-04-18Moved cleanup before raising (the code couldn't be reached before)Massimiliano Culpo1-3/+3
2019-04-18Cleaned get methods of Repo and RepoPathMassimiliano Culpo1-5/+3
2019-04-18Used functools.wrap for the decorator + reordered importsMassimiliano Culpo1-20/+23
2019-04-18Removed 'namespace' argument from Repo and RepoPathMassimiliano Culpo1-32/+19
fixes #11159 The 'namespace' argument to both Repo and RepoPath were used to set the "super namespace". Currently it seems to be vestigial as the only "super namespace" allowed for packages is 'spack.pkg' since 39c9bbf