summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2019-07-20tests: speed up tests that rely on the database fixture (#12031)Massimiliano Culpo6-61/+95
The database and mutable_database fixtures were installing and uninstalling the same specs multiple times to ensure the database for tests has the correct state. This commit optimizes the procedure by caching the state in an external directory, and copying it in instead of going through the installation or uninstallation again. The database fixture is meant not to be modified by tests. This commit enforces this invariant by making the database read-only before starting the test. * Added missing db markers to tests * Added test for uninstall_by_spec * `database` fixture now returns a read-only database * Tests that modify the DB now use `mutable_database` fixture
2019-07-20environment views: refactor stripping build deps (#12069)Greg Becker1-3/+1
2019-07-20compilers: don't raise errors for duplicate compiler definitions (#11910)Greg Becker2-1/+29
Summary: - Allow multiple definitions of compiler in compilers.yaml (use first instance) - Still print debug messages when there are duplicates, to assist users in finding this issue. Merging configs from different scopes can result in multiple compiler being present in the same configuration list. Instead of raising when there are duplicates, take the one with highest precedence. Print a debug message instead of raising, so that we can still diagnose this. We don't have a good way of warning the user about inconsistent configuration *in the same file* -- we'd need to dig into YAML file/line info for that.
2019-07-20environments: add activate/deactivate tests, work wtih `set -u`Todd Gamblin2-6/+54
- [x] Add shell tests to ensure that `spack env activate`, `spack env deactivate`, and `despacktivate` continue to work. - [x] Also ensure that activate and deactivate both work with `set -u`
2019-07-19Package inheritance: Find patch files defined in parent classes using MRO ↵Greg Becker3-1/+40
(#12051) Fixes #8908, 11844 Use Python MRO to find patch files from parent classes.
2019-07-19Fix #11240 (#11995)albestro2-1/+37
* extends mkdirs with permissions for intermediate folders Does not use os.makedirs mode parameter because its behavior is changed with Python 3.7 (it ignores it for intermediate dirs), and moreover it was not possible to set different modes for newly-created folders and leaf folder. reference: - https://bugs.python.org/issue19930 - https://docs.python.org/3.7/library/os.html#os.makedirs * comment mkdirp step easing code understanding * revert mkdir to default for package metapath since metapath is nested in package folder, there is no need to specify permissions for intermediate folders because the prefix already exists. * comment create_install_directory package modes
2019-07-18concretization: fix transient hang in python 3.5Gregory Becker1-1/+2
Bug relates to the interplay between: 1. random dict orders in python 3.5 2. bugfix in initial implementation of stacks for `_concretize_dependencies` when `self._dependencies` is empty 3. bug in coconcretization algorithm computation of split specs Result was transient hang in coconcretization. Fixed #3 (bug in coconcretization) to resolve.
2019-07-18stacks: add option to link only roots or all specs, default to allGregory Becker3-37/+134
2019-07-18stacks: refactor view descriptors into a separate objectGregory Becker2-105/+123
2019-07-18refactor: clean up Environment classGregory Becker2-74/+65
- remove redundant code in Environment.__init__ - use socket.gethostname() instead of which('hostname') - refactor updating SpecList references - refactor 'specs' literals to a single variable for default list name - use six.string_types for python 2/3 compatibility
2019-07-18bugfix: spec lists can now include multiple referencesGregory Becker2-3/+3
2019-07-18stacks: use new format string syntax in environments codeGregory Becker4-11/+12
- Change old spec expressions to use Spack's new spec formatting sytnax.
2019-07-18docs: Add documentation for views in environmentsGregory Becker2-3/+145
2019-07-18stacks: update view management for multiple/combinatorial viewsGregory Becker7-81/+404
This adds notion of a default view, other views in environments
2019-07-18stacks: environment add/remove respect multiply-defined listsGregory Becker2-7/+53
2019-07-18stacks: allow force-removing from a speclist used in a matrixGregory Becker2-12/+21
2019-07-18stacks: update environment add/remove algorithmsGregory Becker3-59/+74
2019-07-18concretization: regression test for constrain() with depsGregory Becker1-0/+1
- ensure that `Spec('foo').constrain('foo ^bar')` works - prior to stacks implementation, this constraint would have done nothing.
2019-07-18docs: initial documentation for environmentsGregory Becker2-0/+636
2019-07-18stacks: initial implementation of stacks on environmentsGregory Becker9-61/+875
- stack syntax in env schema - switch environment specs over to SpecList object - add stack functionality to environments - handle definition extensions through stack.yaml and SpecList - implement conditional definitions - tests
2019-07-17Add support for .txz and .tbz2 file extensions (#12048)Adam J. Stewart3-1/+46
2019-07-17tests: uninstall_by_spec error and rpath_args tests (#11971)Tamara Dahlgren2-0/+26
* tests: Add uninstall_by_spec error tests. These tests were originally included in PR #11797.
2019-07-17feature: Allow developers to use Spack for partial builds (#12006)Greg Becker1-1/+5
Added new diy option.
2019-07-16Ignore Modules v4 environment variables in `from_sourcing_file` (#10753)Massimiliano Culpo3-117/+311
* from_sourcing_file: fixed a bug + added a few ignored variables closes #7536 Credits for this change goes to mgsternberg (original author of #7536) The new variables being ignored are specific to Modules v4. * Use Spack Executable in 'EnvironmentModifications.from_sourcing_file' Using this class avoids duplicating lower level logic to decode stdout and handle non-zero return codes * Extracted a function that returns the environment after sourcing files The logic in `EnvironmentModifications.from_sourcing_file` has been simplified by extracting a function that returns a dictionary with the environment one would have after sourcing the files passed as argument. * Further refactoring of EnvironmentModifications.from_sourcing_file Extracted a function that sanitizes a dictionary removing keys that are blacklisted, but keeping those that are whitelisted. Blacklisting and whitelisting can be done on literals or regex. Extracted a new factory that creates an instance of EnvironmentModifications from a diff of two environments. * Added unit tests * PS1 is blacklisted + more readable names for some variables
2019-07-15Fix build_jobs exceeding number of coresMichael Kuhn1-0/+1
All documentation mentions that `build_jobs` is limited by the number of cores available in the system. This is also enforced when setting it via `--jobs`. However, when setting it via `config.yaml`, it can exceed the number of cores available, making builds run out of memory.
2019-07-15Add auto-dispatch specification to Intel packages (#11697)Glenn Johnson1-1/+33
This PR adds the ability to specify the auto-dispatch targets that can be used by the Intel compilers. The `-ax` flag will be written to the respective compiler configuration files. This ability is very handy when wanting to build optimized builds for various architectures. This PR does not set any optimization flags, however.
2019-07-15spack uninstall can uninstall specs with multiple roots (#11977)Massimiliano Culpo2-14/+52
Fixes #3690 Fixes #5637 Uninstalling dependents of a spec was relying on a traversal of the parents done by inspecting spec._dependents. This is in turn a DependencyMap that maps a package name to a single DependencySpec object (an edge in the DAG) and cannot thus model the case where a spec has multiple configurations of the same parent package installed (for example if different versions of the same Python library depend on the same Python installation). This commit works around this issue by constructing the list of specs to be uninstalled in an alternative way, and adds tests to verify the behavior. The core issue with DependencyMap is not resolved here.
2019-07-12find_libraries searches lib and lib64 before prefix (#11958)Tim Fuller2-15/+23
The default library search for a package checks the lib/ and lib64/ directories for libraries before the root prefix, in order to save time when searching for libraries provided by externals (which e.g. may have '/usr/' as their root). This moves that logic into the "find_libraries" utility method so packages implementing their own custom library search logic can benefit from it. This also updates packages which appear to be replicating this logic exactly, replacing it with a single call to "find_libraries".
2019-07-12Copy --{use,no}-cache from install to bootstrap (#11786)Oliver Breitwieser1-1/+10
2019-07-12'spack compiler add' resolves relative path to absolute path (#11792)Carson Woods1-0/+1
Fixes #11782 Spack was not properly resolving relative paths to absolute paths when a relative path was passed to "spack compiler add [PATH]". Now, if provided a relative path, the absolute path is written to compilers.yaml rather than the relative path.
2019-07-12Add template creation test, --skip-editor option (#11970)Tamara Dahlgren2-2/+63
* Add template creation test * Added --skip-editor option to "spack create": normally "spack create" opens an editor for the user after generating a package file; when the --skip-editor option is used, "spack create" only generates the package file and does not open an editor * Added --skip-editor option to bash completion
2019-07-12Fix pic_flag and test contents about Fujitsu compiler. (#11996)t-karatsu2-6/+6
2019-07-12bugfix: show dot after namespaces in `spack find -N` (#11945)Todd Gamblin2-1/+14
- Namepsaces were shown without dots after the new format strings were added. - Add a test for `spack find` to ensure that find -N shows the right output.
2019-07-11test: Extra possible-dependencies coverage check (#11988)Tamara Dahlgren1-1/+10
2019-07-11Rename build logs and make names consistent (#11806)Tamara Dahlgren9-92/+228
Fixes #11781 * Rename build log to spack-build-log.txt * Rename environment variables file to spack-build-env.txt * The name of the log and env files is now the same during the build and after the build completes * Update packages which referred to the build log/env files * For packages installed before this commit using older names for the build and env files, search for the older names
2019-07-10tutorial: correct image link on tutorial pageTodd Gamblin1-1/+1
2019-07-10tutorial: updates slides for DOE/NSF workshop.Todd Gamblin1-9/+8
2019-07-08bugfix: conflicts should print out the spec that has the conflict (#11947)Todd Gamblin1-1/+6
- Fix a bug introdcued by removing parse_anonymous_spec() - Conflicts' when specs are now *actually* anonymous, and the name of the package is implicit, so we need to remember to add it back to error messages.
2019-07-05bugfix: always generate a stack trace when spack is run with --debug (#11940)Todd Gamblin1-0/+5
- We weren't previously printing stack traces on SystemExit or KeyboardInterrupts. - Either raise or print the stacktrace in these cases.
2019-07-01specs: remove parse_anonymous_spec(); use Spec() insteadTodd Gamblin6-144/+168
- `parse_anonymous_spec()` is a vestige of the days when Spack didn't support nameless specs. We don't need it anymore because now we can write Spec() for a spec that will match anything, and satisfies() semantics work properly for anonymous specs. - Delete `parse_anonymous_spec()` and replace its uses with simple calls to the Spec() constructor. - make then handling of when='...' specs in directives more consistent. - clean up Spec.__contains__() - refactor directives and tests slightly to accommodate the change.
2019-07-01cray: use the cle-release file to determine CNL versionTodd Gamblin2-6/+72
- CNL OS previously used the *Cray PE* version to determine the OS version. Cray does not synchronize PE and CLE releases; you can run CLE7 with PrgEnv 6 (and NERSC currently does). - Fix Spack's OS detection by using the cle-release file to detect the OS version. This file is updated with every CLE OS release. - Add some tests for our parsing logic
2019-07-01buildcache: add replace_prefix_bin function to deal with hard coded paths in ↵Patrick Gartung1-8/+27
libraries and executables. (#11882)
2019-06-30style: remove unnecessary NOQA exceptionsTodd Gamblin1-9/+9
2019-06-29Allow uninstalling missing packages (#11874)Peter Scheibel2-6/+19
Remove package access from directory_layout; add regression test to ensure that specs can be uninstalled without a package being known
2019-06-28docs: add example for an external module in packages.yamlMark W. Krentel1-2/+12
Add an example of a 'modules:' entry for an external package in packages.yaml. The 'External Packages' section of 'Build Customization' mentions 'paths:' and 'modules:' and gives an example of paths, but not modules.
2019-06-26Add --frontend and --backend option to spack arch command (#11746)Toyohisa Kameyama2-2/+26
2019-06-24Fix for #10063 : install from source if buildcache with different DAG exist ↵Patrick Gartung1-1/+6
(#11834) * Fix for #10063 : install from source if buildcache with different DAG exist * Flake8
2019-06-21Use Stage.archive_file to access non-expanded download (#11817)Peter Scheibel2-4/+25
Fixes #11816 Allow packages to refer to non-expanded downloads (e.g. a single script) using Stage.archive_file. This addresses a regression from #11688 and adds a unit test for it.
2019-06-20Ignore pgcc-llvm and friends, default to pgcc (#11703)Adam J. Stewart1-4/+0
This change reverts to the previous behavior of only looking for pgcc and friends, not pgcc-llvm and friends. The llvm variant doesn't support all the same features as the traditional variant of the pgi code generator; this change avoids treating the llvm variant as a default pgi compiler. This retains the changes in #10704 which accept the "LLVM" suffix of the version string of the PGI compiler, which allows users to explicitly add the llvm-pgi compiler if desired.
2019-06-20Resources: use expanded archive name by default (#11688)Peter Scheibel4-45/+135
For resources, it is desirable to use the expanded archive name of the resource as the name of the directory when adding it to the root staging area. #11528 established 'spack-src' as the universal directory where source files are placed, which also affected the behavior of resources managed with Stages. This adds a new property ('srcdir') to Stage to remember the name of the expanded source directory, and uses this as the default name when placing a resource directory in the root staging area. This also: * Ensures that downloaded sources are archived using the expanded archive name (otherwise Spack will not be able to determine the original directory name when using a cached archive). * Updates working_dir context manager to guarantee restoration of original working directory when an exception occurs * Adds a "temp_cwd" context manager which creates a temporary directory and sets it as the working directory