summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2019-05-16Command extensions can import code from modules in root or cmd folder (#11209)Massimiliano Culpo4-4/+277
#8612 added command extensions to Spack: a command implemented in a separate directory. This improves the implementation by allowing the command to import additional utility code stored within the established directory structure for commands. This also: * Adds tests for command extensions * Documents command extensions (including the expected directory layout)
2019-05-17Allow command access to dump/pickle_environment from #8476 (#11434)Chris Green2-7/+56
* Allow command access to dump/pickle_environment from #8476
2019-05-15fix suite-sparse built with tbb from intel-parallel-studio (#11134)Denis Davydov1-1/+11
* fix suite-sparse built with tbb from intel-parallel-studio * intel: add tbb_headers, strip newline from cxx_lib * use property
2019-05-15Use `svn info --xml` instead of `svn info` to get svn revisions (#11466)Todd Gamblin2-14/+8
- `svn info` prints different results depending on the system locale - in particular, Japanese output doesn't contain "Revision:" - Change Spack code to use XML output instead of using the human output
2019-05-14Remove vestigial print statement in module logic (#11438)Greg Becker1-1/+0
Remove a vestigial print statement introduced in #8570
2019-05-10permissions: preserve suid and sgid bits (#10727)Greg Becker2-0/+46
* Don't overwrite suid/sgid bits when setting permissions * add tests for permission setting
2019-05-10Intel packages: multiple installs and optional scalapack libs (#11384)Glenn Johnson1-1/+5
Add fixes to support multiple installs and dependents using a subset of IntelPackage functionality. * Update IntelPackage to only return scalapack libraries if the root spec depends on MPI: scalapack requires MPI to be mentioned as a dependency in the DAG. Package builds using intel-mkl for its blas/lapack implementations but not for scalapack were failing to build. Ideally it would be possible to ask if any of the packages in the DAG are actually requesting the scalapack functionality provided by the IntelPackage and only return scalapack libs in that case, but that is not easily done at this time. Fixes #11314 Fixes #11289 * set HOME when the intel silent installer is run. This prevents the installer from using the ~/intel directory (which can cause conflicts for multiple installs of the same IntelPackage) Fixes #9713
2019-05-09modules: use new `module` function instead of `get_module_cmd` (#8570)Greg Becker8-248/+118
Use new `module` function instead of `get_module_cmd` Previously, Spack relied on either examining the bash `module()` function or using the `which` command to find the underlying executable for modules. More complicated module systems do not allow for the sort of simple analysis we were doing (see #6451). Spack now uses the `module` function directly and copies environment changes from the resulting subprocess back into Spack. This should provide a future-proof implementation for changes to the logic underlying the module system on various HPC systems.
2019-05-10make license check more-strictPeter Josef Scheibel2-39/+93
The license text is now expected to match almost exactly (not accounting for formatting in different file types (e.g. rst vs. bash script vs. python)
2019-05-08Hide some variables in spack-build.env fileSeth R Johnson1-2/+6
Don't arbitrarily reset PWD and OLDPWD when sourcing, as well as other session-specific IDs
2019-05-07Environment path sanitization and sourcing (#8476)Chris Green3-7/+208
Add two functions to the EnvironmentModifications object to help users sanitize environment variables in their package definitions: * deprioritize_system_paths: this keeps system paths in the environment variable but moves them to the end. * prune_duplicate_paths: remove any duplicate paths from the variable This includes testing for the new functions as well as for (previously-untested) old convenience functions for environment variable manipulation. This also adds special handling for bash functions so they will be defined when the exported environment file is sourced.
2019-05-04Binary caches on MacOS - allow expanded RPATHs (#11345)Patrick Gartung2-0/+37
Fixes #11335 Update the Spack compiler wrappers to add the headerpad_max_install_names linker flag on MacOS. This allows the install_name_tool to rewrite the RPATH entry of the binary to be longer if needed. This is primarily useful for creating and distributing binary caches of packages (i.e. using the "spack buildcache" command); binary caches created on MacOS before this commit may not successfully relocate (if the target root path is larger).
2019-05-04Added a function that concretizes specs together (#11158)Massimiliano Culpo3-2/+112
* Added a function that concretizes specs together * Specs concretized together are copied instead of being referenced This makes the specs different objects and removes any reference to the fake root package that is needed currently for concretization. * Factored creating a repository for concretization into its own function * Added a test on overlapping dependencies
2019-05-03extend Version class so that 2.0 > 1.develop > 1.1 and develop > master > ↵Denis Davydov4-72/+123
head > trunk > 9999 (#1983) * extend Version class so that 2.0 > 1.develop > 1.1 * add concretization tests, with preferences and preferred version. * add master, head, trunk as develop-like versions, develop > master > head > trunk * update documentation on version comparison
2019-05-02fix error when adding package to env in container (#11321)Levi Baber1-1/+1
2019-05-02Don't use buildcache to install patchelf (#11343)Patrick Gartung1-1/+1
2019-05-01Fix spack flake8 to use Travis's target as base when neededTodd Gamblin1-2/+6
2019-04-24_valid_tokens list is missing intended comma (#11271)Daryl W. Grunau1-1/+1
* _valid_tokens list is missing a needed comma
2019-04-23Update tutorial page with RIKEN tutorial slides. (#11254)Todd Gamblin1-13/+28
- Add link to container image, as well as a description of VMs - Update slide link to point to latest RIKEN tutorial
2019-04-22Fix transitions between tutorial sections (#11251)Greg Becker3-11/+21
2019-04-20spack edit: use execv instead of Executable (#11245)Todd Gamblin3-37/+262
- `spack edit` previously used `spack.util.executable` `Executable` objects, and didn't `exec` the editor like you'd expect it to - This meant that Spack was still running while your editor was, and stdout/stdin were being set up in weird ways - e.g. on macOS, if you call `spack edit` with `EDITOR` set to the builtin `emacs` command, then type `Ctrl-g`, the whole thing dies with a `==> Error: Keyboard interrupt` - Fix all this by changing spack.util.editor to use `os.execv` instead of Spack's `Executable` object
2019-04-20Fix detection of LLVM-enabled PGI compilers (#10704)Adam J. Stewart2-2/+12
* Fix detection of LLVM-enabled PGI compilers * Add unit tests for LLVM-enabled PGI compiler version detection
2019-04-20Move NoLibrariesError/NoHeadersError into error.py (#10997)Denis Davydov3-11/+20
Also add constructor to NoLibrariesError which can either take an error message (like other SpackErrors) or a name and prefix (in which case the error message is constructed).
2019-04-19Fix outdated R packages failing to fetch (#11039)Justin S1-3/+4
PR #10758 made a slight change to find_versions_of_archive() which included archive_url in the search process. While this fixed `spack create` and `spack checksum` missing command-line arguments, it caused `spack install` to prefer those URLs over those it found in the scrape process. As a result, the package url was treated as a list_url causing all R packages to stop fetching once the package was updated on CRAN. This patch is more selective about including the archive_url in the remote versions, explicitly overriding it with matching versions found by the scraper.
2019-04-19Fix backwards compatibility for module naming (#11236)Greg Becker1-0/+4
f242f5f8 changed the format strings but maintained backwards compatibility in all cases except one: The list of valid tokens for the module naming schemes was not updated properly to contain both the new and old styles for compilers and package names. This PR re-adds the old tokens into the list of valid tokens.
2019-04-18Spack chain docs: config should go in upstreams.yaml (#11225)Peter Scheibel1-14/+12
#11152 added documentation for #8772 but some details were based on an earlier implementation that had changed by the time #8772 was merged. In particular, #11152 mentioned that upstream Spack instances were configured in config.yaml, when in fact they should be placed in a separate upstreams.yaml config file; this PR updates the documentation accordingly.
2019-04-18Moved cleanup before raising (the code couldn't be reached before)Massimiliano Culpo1-3/+3
2019-04-18Cleaned get methods of Repo and RepoPathMassimiliano Culpo1-5/+3
2019-04-18Used functools.wrap for the decorator + reordered importsMassimiliano Culpo1-20/+23
2019-04-18Removed 'namespace' argument from Repo and RepoPathMassimiliano Culpo1-32/+19
fixes #11159 The 'namespace' argument to both Repo and RepoPath were used to set the "super namespace". Currently it seems to be vestigial as the only "super namespace" allowed for packages is 'spack.pkg' since 39c9bbf
2019-04-18improvements to our CDash reporter (#11168)Zack Galbreath2-84/+127
* Make a separate CDash report for each package installed Previously, we generated a single CDash report ("build") for the complete results of running a `spack install` command. Now we create a separate CDash build for each package that was installed. This commit also changes some of the tests related to CDash reporting. Now only one of the tests exercises the code path of uploading to a (nonexistent) CDash server. The rest of the related tests write their reports to disk without trying to upload them. * Don't report errors to CDash for successful packages Convert errors detected by our log scraper into warnings when the package being installed reports that it was successful. * Report a maximum of 50 errors/warnings to CDash This is in line with what CTest does. The idea is that if you have more than 50 errors/warnings you probably aren't going to read through them all anyway. This change reduces the amount of data that we need to transfer and store.
2019-04-17Features: Improve Spec format strings (#10556)Greg Becker39-177/+561
* Update spec format to simpler syntax, maintain backwards compatibility * Switch to new spec.format method throughout internals * update package files for new format strings * documentation and minor code cleanup. removed nonsensical variant sigils
2019-04-17Add ftn to lib/spack/env (#11180)Satish Balay1-0/+1
Fixes #11070 #11010 Spack attempts to intercede on behalf of all compiler invocations for a build. This involves adding its wrappers to PATH. Cray systems include a "ftn" executable and Spack was only redirecting this call when the Spec was built with cce. This updates the compiler wrappers to add "ftn" in all cases.
2019-04-16don't record 'view: True' in environment config (#11182)Peter Scheibel1-1/+8
The default (implied) behavior for all environments, as of ea1de6b, is that an environment will maintain a view in a location of its choosing. ea1de6b explicitly recorded all three possible states of maintaining a view: 1. Maintain a view, and let the environment decide where to put it (default) 2. Maintain a view, and let the user decide 3. Don't maintain a view This commit updates the config writer so that for case [1], nothing will be written to the config.yaml. This will not change any existing behavior, it just serves to keep the config more compact.
2019-04-16Add unit tests for Cray compiler detection (#11191)Adam J. Stewart1-0/+11
2019-04-15Bugfix: Install missing compilers for dependency packages (#11175)Greg Becker1-9/+8
Compilers are treated separately from other dependencies in Spack. #10761 added the option to automatically install compilers when a package specifies using a compiler that is not available in Spack. However, this did not work correctly for dependency packages (it would only build a compiler for the root of an install DAG). This commit enables the building of compilers for dependency packages.
2019-04-11Updated Sphinx configuration (#11165)Massimiliano Culpo1-3/+3
2019-04-11intel: optionally take gcc executable from cflags (#11136)Denis Davydov1-3/+15
2019-04-10Maintain a view for an environment (#10017)Peter Scheibel7-114/+569
Environments are nowm by default, created with views. When activated, if an environment includes a view, this view will be added to `PATH`, `CPATH`, and other shell variables to expose the Spack environment in the user's shell. Example: ``` spack env create e1 #by default this will maintain a view in the directory Spack maintains for the env spack env create e1 --with-view=/abs/path/to/anywhere spack env create e1 --without-view ``` The `spack.yaml` manifest file now looks like this: ``` spack: specs: - python view: true #or false, or a string ``` These commands can be used to control the view configuration for the active environment, without hand-editing the `spack.yaml` file: ``` spack env view enable spack env view envable /abs/path/to/anywhere spack env view disable ``` Views are automatically updated when specs are installed to an environment. A view only maintains one copy of any package. An environment may refer to a package multiple times, in particular if it appears as a dependency. This PR establishes a prioritization for which environment specs are added to views: a spec has higher priority if it was concretized first. This does not necessarily exactly match the order in which specs were added, for example, given `X->Z` and `Y->Z'`: ``` spack env activate e1 spack add X spack install Y # immediately concretizes and installs Y and Z' spack install # concretizes X and Z ``` In this case `Z'` will be favored over `Z`. Specs in the environment must be concrete and installed to be added to the view, so there is another minor ordering effect: by default the view maintained for the environment ignores file conflicts between packages. If packages are not installed in order, and there are file conflicts, then the version chosen depends on the order. Both ordering issues are avoided if `spack install`/`spack add` and `spack install <spec>` are not mixed.
2019-04-10Unload altd and darshan (#11162)Mario Melara1-1/+1
2019-04-10Add documentation for chaining Spack instances (#11152)Peter Scheibel3-0/+102
Add documentation for the Spack chain feature added in #8772
2019-04-09Typo: Unkown -> Unknown (#11150)George Hartzell1-1/+1
2019-04-09intel: consolidate libs() in the base class (#11143)Denis Davydov1-3/+10
* intel: consolidate libs() in the base class * white space * flake8
2019-04-06Allow spack install to take either cdash stamp or track (#11106)Scott Wittenburg3-4/+44
When providing a track, the cdash reporter will format the stamp itself, as it has always done, and register the build during the package installation process. When providing a stamp, it should first be formatted as cdash expects, and then cdash will be sure to report results to same build id which was registered manually elsewhere.
2019-04-05Fix reading externals from old databases (#11118)Greg Becker4-2/+36
* Update Spec.prefix to have special case for 'None' in database path; regression test * Update in database reader rather than spec * Change assertion to conditional + raise * Added test for concrete check in Spec.prefix
2019-04-05Fix module_parsing test (#11087)Michael Kuhn1-0/+3
The module_parsing test checks whether the module function is available by looking for the string 'not found'. If the user has set a different locale, the test can assume that the module function is available when it actually is not.
2019-04-02Fix directory layout using namespace (#11076)Tim Fuller2-1/+8
Directory layouts using ${NAMESPACE} were broken. This addresses that and adds a test to prevent regression.
2019-04-02spack repo add: fix error message when `packages' directory is missing (#11031)Tristan Carel1-1/+1
2019-04-02Added tests on compiler's version detection (#10931)Massimiliano Culpo12-258/+200
* Split get_compiler_version into two functions: get_compiler_version_output runs the compiler with the relevant option to print the version; extract_version_from_output determines the version by examining this output. This makes it easier to test the customized version detection for each compiler. Users can customize this by overriding the following: * version_argument: this is the argument that tells the compiler to print its version. It assumes that the compiler will report its version if invoked with a single option (like "--version") * version_regex: the regular expression used to extract the version from the compiler argument. This assumes that a regular expression is sufficient to extract the version, and that the version can be extracted from a single capture group (Spack uses the first capture group) * default_version: allows you to completely override all version detection logic * get_compiler_version_output: if getting the compiler to report its version is more complex than invoking it with a single arg * extract_version_from_output: if it is difficult to define a regex that can be used to extract the version from the output * Added tests for version detection of most compilers * Removed redundant code from xl_r compiler class (by inheriting from xl compiler definition)
2019-03-29Update llnl.util.lang.memoized so that Sphinx can extract signature (#11055)Massimiliano Culpo1-18/+12
Replace the original implementation of the "memoized" decorator with an implementation that exposes the docstring and arguments of the wrapped function. This is achieved using functools.wraps.