summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2019-06-21Use Stage.archive_file to access non-expanded download (#11817)Peter Scheibel2-4/+25
Fixes #11816 Allow packages to refer to non-expanded downloads (e.g. a single script) using Stage.archive_file. This addresses a regression from #11688 and adds a unit test for it.
2019-06-20Ignore pgcc-llvm and friends, default to pgcc (#11703)Adam J. Stewart1-4/+0
This change reverts to the previous behavior of only looking for pgcc and friends, not pgcc-llvm and friends. The llvm variant doesn't support all the same features as the traditional variant of the pgi code generator; this change avoids treating the llvm variant as a default pgi compiler. This retains the changes in #10704 which accept the "LLVM" suffix of the version string of the PGI compiler, which allows users to explicitly add the llvm-pgi compiler if desired.
2019-06-20Resources: use expanded archive name by default (#11688)Peter Scheibel4-45/+135
For resources, it is desirable to use the expanded archive name of the resource as the name of the directory when adding it to the root staging area. #11528 established 'spack-src' as the universal directory where source files are placed, which also affected the behavior of resources managed with Stages. This adds a new property ('srcdir') to Stage to remember the name of the expanded source directory, and uses this as the default name when placing a resource directory in the root staging area. This also: * Ensures that downloaded sources are archived using the expanded archive name (otherwise Spack will not be able to determine the original directory name when using a cached archive). * Updates working_dir context manager to guarantee restoration of original working directory when an exception occurs * Adds a "temp_cwd" context manager which creates a temporary directory and sets it as the working directory
2019-06-19fixed spec dependence attribute writing. Fixes #11686 (#11776)Gregory Lee1-1/+1
2019-06-18Remove test dependency on /usr/bin/gcc (#11769)Greg Becker1-3/+10
The regression test for #11678 fails on at least some Mac OS systems because they have a /usr/bin/gcc that is secretly clang. This PR replaces the dependency on a system gcc executable with a test-generated script that generates the expected output for the compiler logic.
2019-06-18Rename build.out to build.txt (#11749)Elizabeth Fischer3-5/+5
Rename build.out to build.txt; makes it easier when people post log files to the Spack mailing list.
2019-06-17bugfix: use config fixture for stage tests (#11701)Tamara Dahlgren2-6/+23
Some tests introduced in #11528 temporarily set the user's `config:build_stage`, which affected (or created) a config.yaml file in the user's `$HOME/.spack` directory that could leave entries behind if the tests fail. This change ensures only temporary configuration files are used/affected by these tests.
2019-06-17docs: updates for `stage.source_path` (#11702)Tamara Dahlgren3-13/+37
2019-06-16docs: update tutorial page for ISC19 tutorial (#11742)Todd Gamblin1-10/+13
2019-06-15docs: fix config tutorial (#11715)Michael Kuhn1-30/+35
This fixes a few typos, adds a hint to `spack config blame` and brings the variant specification in line with other docs sections.
2019-06-13Ensure mock configuration directory is cleaned up on session exit. (#11700)Tamara Dahlgren1-1/+4
2019-06-13Add tests for "spack location" command (#11661)Tamara Dahlgren2-0/+135
The "spack location" command was previously untested. This also adds a check to ensure that composite Stages can report whether they were expanded (this property was previously only recorded in Stage but not in CompositeStage).
2019-06-13Always treat DIYStage as expanded (#11663)Tamara Dahlgren2-2/+73
DIYStage, used to treat a user-managed directory as a staging area, should always be considered expanded (i.e. the source has been decompressed if it was stored in an archive). This also: * Adds checks to ensure that the path used to instantiate a DIYStage refers to an existing directory. * Adds tests to check the behavior of DIYStage (including behavior added here, but it was generally untested before).
2019-06-12Update remaining packages to use Stage.source_path (#11662)Tamara Dahlgren2-3/+3
#11528 updated Stage to always store a Package's source in a fixed directory accessible via `Stage.source_path` This left behind a number of packages which were expecting to access the source code via `Stage.path`. This Updates those packages to use `Stage.source_path` instead. This also updates the name of the fixed directory: The original name of the fixed directory was "src", so if an expanded archive created a "src" directory, then users inspecting the directory structure could see paths like "src/src" (which wasn't wrong but could be confusing). Therefore this also updates the name of the fixed directory to "spack-src".
2019-06-12Make "spack compiler find" check PATH by default (#11683)Massimiliano Culpo5-21/+60
Fixes #11678 `spack compiler find` was not searching `PATH` when provided with no arguments. ea7910a updated the API for the search function and the command logic did not update how it called this function. This also adds a test to ensure that `spack compiler find` will collect compilers from `PATH`.
2019-06-11Add additional common C++ and Fortran header file extensions (#11600)Adam J. Stewart1-1/+8
* Add additional common C++ and Fortran header file extensions * Add .hxx extension * Add .txx and .tcc extensions * Add .icc extension
2019-06-11Add extra logic for C std flags on PGI, XL, and Cray (#11635)Chuck Atkins3-7/+57
2019-06-10Fix recursive module find for upstream dependencies (#11304)Peter Scheibel4-64/+223
"spack module tcl find -r <spec>" (and equivalents for other module systems) was failing when a dependency was installed in an upstream Spack instance. This updates the module index to handle locating module files for upstream Spack installations (encapsulating the logic in a new class called UpstreamModuleIndex); the updated index handles the case where a Spack installation has multiple upstream instances. Note that if a module is not available locally but we are using the local package, then we shouldn't use a module (i.e. if the package is also installed upstream, and there is a module file for it, Spack should not use that module). Likewise, if we are instance X using upstreams Y and Z like X->Y->Z, and if we are using a package from instance Y, then we should only use a module from instance Y. This commit includes tests to check that this is handled properly.
2019-06-08concretization: don't apply build-dep constraints for installed packages ↵Peter Scheibel3-5/+85
(#11594) Spack currently tries to unify everything in the DAG, but this is too strict for build dependencies, where it is fine to build a dependency with a tool that conflicts with a version fo that tool for a dependent's build. To enable a workaround for conflicts among build dependencies, so that users can install in multiple steps to avoid these conflicts, make the following changes: * Dont apply package dependency constraints for build deps of installed packages * Avoid applying constraints for installed packages vs. concrete packages * Mark all dependencies of installed packages as visited in normalization method * don't remove dependency links for concrete specs in flat_dependencies Also add tests: * Update test to ensure that link dependencies of installed packages have constraints applied * Add test to check for proper handling of transitive dependencies (which is currently not the case)
2019-06-07spack list: latest version (JSON) (#11652)Axel Huebl4-1/+81
List the latest version of each package in JSON encoding. Preparation for consumption for a "spack badge" service.
2019-06-07Compiler search uses a pool of workers (#10190)Massimiliano Culpo17-302/+503
- spack.compilers.find_compilers now uses a multiprocess.pool.ThreadPool to execute system commands for the detection of compiler versions. - A few memoized functions have been introduced to avoid poking the filesystem multiple times for the same results. - Performance is much improved, and Spack no longer fork-bombs the system when doing a `compiler find`
2019-06-06commands: remove unused `spack list --format=rst` (#11651)Todd Gamblin2-79/+0
- We use `spack list --foramt=html` now, as it is much faster and doesn't make the docs build take forever. - Remove `spack list --format=rst` as it is no longer used.
2019-06-06bugfix: StageComposite must include the `expanded` property` (#11647)Todd Gamblin1-0/+4
2019-06-06tests: package_sanity now checks packages in 3rd-party repos (#11637)Jennifer Herting1-2/+1
2019-06-05tests: add extra coverage for fetch strategy testsTamara Dahlgren4-8/+142
2019-06-05stage: make `source_path` available before stage is builtTamara Dahlgren6-207/+535
- `stage.source_path` was previously overloaded; it returned `None` if it didn't exist and this was used by client code - we want to be able to know the `source_path` before it's created - make stage.source_path available before it exists. - use a well-known stage source path name, `$stage_path/src` that is available when `Stage` is instantiated but does not exist until it's "expanded" - client code can now use the variable before the stage is created. - client code can test whether the tarball is expanded by using the new `stage.expanded` property instead of testing whether `source_path` is `None` - add tests for the new source_path semantics
2019-06-05refactor: remove unused spack.stage._get_mirrors() functionTamara Dahlgren1-37/+24
2019-06-05tty: make tty.* print exception typesTamara Dahlgren15-26/+55
- make tty.msg, tty.info, etc. print the exception type and stringified message if the message argument is an exception. - simplify parts of the code that call tty.debug(str(e)) - add extra tty.debug statements in places where exceptions were previously ignored
2019-06-05clang: fix error messages in c11_flag, cxx17_flag (#11632)Justin S1-2/+2
2019-06-05graph: refactor static graphsTodd Gamblin4-100/+73
- `spack graph --static` (and `spack.graph.dot_graph`) now do the "right thing" and print the possible dependency graph of provided packages. - `spack graph --static` no longer concretizes specs, as it only relies on class level metadata - Previously the behavior was not consistent -- `spack graph --static` would graph possible dependencies of concrete specs, but would only include some of them. The new code properly pursues all possible dependencies, and allows traversing by different dependency types.
2019-06-05spack dependencies: support --deptype argumentTodd Gamblin4-10/+47
- `spack dependencies` can now take a --deptype argument to only traverse particular deptypes - add a new "common" argument for deptype in spack.cmd.common.arguments - Database.installed_relatives() can now also take a deptype argument - this is used by `spack dependencies --installed`
2019-06-05package: make possible_dependencies consider deptypesTodd Gamblin3-16/+93
- `PackageBase.possible_dependencies` now: - accepts a deptype param that controls dependency types traversed - returns a dict mapping possible depnames to their immediate possible dependencies (this lets you build a graph easily) - Add tests for PackageBaes
2019-06-05refactor: remove vestigial ALLOWED_URL_SCHEMES from package.pyTodd Gamblin1-4/+0
2019-06-05refactor: make Package.name consistent with other class attributesTodd Gamblin2-11/+18
- The 'name' attribute for packages was being set in DirectiveMeta, which wasn't consistent with other class properties (like fullname, etc.) - Move it to be a class property of `PackageMeta`, and add the corresponding property method wrapper on `PackageBase`
2019-06-05refactor: make PackageBase.possible_dependencies() a class method.Todd Gamblin1-4/+5
2019-06-05add C standard flags to compiler classes (#11618)Justin S7-0/+90
* add c99_flag, c11_flag to compiler class * implement c99_flag, c11_flag for gcc * implement c99_flag, c11_flag for arm * implement c99_flag for cce * implement c99_flag, c11_flag for clang * implement c99_flag, c11_flag for intel * implement c99_flag, c11_flag for xl
2019-06-04modules: set permissions based on package configuration (#11337)Greg Becker5-52/+99
Previously, module files were not set with the same permissions as the package installation. For world-readable packages, this would not cause a problem. For group readable packages, it does: ``` packages: mypackage: permissions: group: mygroup read: group write: group ``` In this case, the modulefile is unreadable by members of the group other than the one who installed it. Add logic to the modulefile writers to set the permissions based on the configuration in `packages.yaml`
2019-06-04Buildcache relocate.py error fix (#11616)Patrick Gartung1-0/+2
* Add a trailing / if file --mime does not return a mimetype with a / in it * Flake8
2019-05-31Build cache: relocate path to spack/bin/sbang in text files. (#11592)Patrick Gartung3-13/+35
* Build cache: relocate path to spack/bin/sbang in text files. * Found in testing. * update packaging test * Make sbang replacement including #!/bin/bash. Add an additional spack prefix replacement to fix stage directory references. * flake8 * Use buildinfo.get() so old buildcaches without buildinfo['spackprefix'] can be read.
2019-05-28Cap the maximum number of build jobs (#11373)Massimiliano Culpo5-16/+49
* config:build_jobs now controls the number of parallel jobs to spawn during builds, but cannot ever exceed the number of cores on the machine. * The default is set to 16 or the number of available cores, whatever is lowest. * Updated docs to reflect the changes done to limit parallel builds
2019-05-27docs: set gettext_uuid to False to reduce churn (#11567)Todd Gamblin2-1/+2
- `gettext_uuid=True` makes every commit update every .pot file in spack/localized-docs, and speeds up the internationalized doc build slightly. - Optimize for less repository churn, and use `python-levenshtein` to accelerate the build instead.
2019-05-27mirror create: non-zero return code (#8585)Axel Huebl1-0/+2
`mirror create` should return a non-zero return code if errors occured.
2019-05-26link_tree: add option to merge link trees with relative targetsTodd Gamblin2-40/+90
- previous version of link trees would only do absolute symlinks - this version can do relative links using merge(relative=True)
2019-05-26docs: make docs build relocatable for localized buildsTodd Gamblin21-50/+59
- make all Spack paths relative to a `_spack_root` symlink, so that we can easily relocate the docs build *outside* lib/spack/docs - set some useful defaults for gettext translation variables in conf.py - update `relativeinclude` and other references to the spack root in the RST files to use _spack_root
2019-05-26docs: remove legacy import for Sphinx 1.7Todd Gamblin1-6/+1
2019-05-26docs: match ReadTheDocs; use -W (warnings as errors) with SphinxTodd Gamblin1-1/+1
2019-05-26commands: add --update option to `spack list`Todd Gamblin5-96/+171
- Add a `--update FILE` option to `spack list` - Output is written to the file only if any package is newer than the file - Simplify the code in docs/conf.py using this new option
2019-05-26commands: Add --header and --update options to `spack commands`Todd Gamblin4-54/+143
The Spack documentation currently hard-codes some functionality in `conf.py`, which makes the doc build less "pluggable" for things like localized doc builds. In particular, we unconditionally generate an index of commands and a package list as part of the docs, but those should really only be done if things are not up to date. This commit does the following: - Add `--header` option to `spack commands` so that it can do the work of prepending text to its output. - Add `--update FILE` option to `spack commands` that makes it generate a new command index *only* if FILE is out of date w.r.t. commands in the Spack source. - Simplify code in `conf.py` to use these options and only update the command index when needed.
2019-05-26docs: Use HDF5 as the example package for `spack graph`Todd Gamblin2-13/+17
2019-05-24build env: simplify handling of parallel jobs (#11524)Massimiliano Culpo9-35/+97
This PR implements several refactors requested in #11373, specifically: - Config scopes are used to handle builtin defaults, command line overrides and package overrides (`parallel=False`) - `Package.make_jobs` attribute has been removed; `make_jobs` remains as a module-scope variable in the build environment. - The use of the argument `-j` has been rationalized across commands - move '-j'/'--jobs' argument into `spack.cmd.common.arguments` - Add unit tests to check that setting parallel jobs works as expected - add new test to ensure that build job setting is isolated to each build - Fix packages that used `Package.make_jobs` (i.e. `bazel`)