summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2018-08-27import spack load error message (#9088)Andreas Baumbach1-1/+1
It now additionally prints spec hash and options
2018-08-27Moved functions returning default scopes to spack.configMassimiliano Culpo7-34/+39
The functions returning the default scope to be modified or listed have been moved from spack.cmd to spack.config. Lmod now writes the guessed core compiler in the default modify scope instead of the 'site' scope.
2018-08-27Spack can guess lmod core compilers, if none is already presentMassimiliano Culpo2-5/+64
closes #8916 Currently Spack ends with an error if asked to write lmod modules files and the 'core_compilers' entry is not found in `modules.yaml`. After this PR an attempt will be made to guess that entry and the site configuration file will be updated accordingly. This is similar to what Spack already does to guess compilers on first run.
2018-08-25Update to latest version of pytest and dependencies (#9087)Adam J. Stewart1-1/+1
* Update to latest version of pytest and dependencies * Fix bug in installation tests
2018-08-25bugfix: 'submodules' attribute should imply git fetchTodd Gamblin2-1/+8
- 'submodules' wasn't properly declared as an optional attribute for git fetcher - add it and add a test.
2018-08-25commands: add `spack url stats` commandTodd Gamblin2-2/+94
This command prints out stats about all package versions, like so: $ spack url stats ==> 6070 total versions for 2827 packages: ------------------------------------ url 5411 89.1% schemes https 3822 70.6% http 1527 28.2% ftp 27 0.5% file 35 0.6% checksums sha512 6 0.1% sha256 163 3.0% sha1 81 1.5% md5 5161 95.4% ------------------------------------ svn 3 0.0% ------------------------------------ hg 5 0.1% ------------------------------------ go 1 0.0% ------------------------------------ git 650 10.7% security no commit 384 59.1% commit 266 40.9% ------------------------------------
2018-08-24Remove Python 3.3 from testing.Todd Gamblin2-5/+5
- Support for Python 3.3 isn't really needed, as nothing uses it as the default system Python, and nearly everyone will have a newer Python 3 version installed.
2018-08-22More consistent colored messages from activate/deactivate (#9036)Adam J. Stewart1-17/+16
* More consistent colored messages from activate/deactivate * Make 'activating' the default, make 'activated' the debug
2018-08-22Added a new target message to comply with newer versions of GNU Make (#9068)Massimiliano Culpo2-0/+3
#fixes 9067 Unit tests were failing on a system with GNU Make v 4.1
2018-08-21tests: add tests for warnings in build outputTodd Gamblin4-8/+85
- add tests for build warnings and error messages - simplify some code in build environment with plural() function
2018-08-21Display warnings if no errors are found in build logAdam J. Stewart1-1/+10
2018-08-21Fix error caused by missing spack-build.out build logAdam J. Stewart2-3/+3
2018-08-20yaml: use ruamel.yaml instead of pyyamlTodd Gamblin67-8129/+5165
- ruamel.yaml allows round-tripping comments from/to files - ruamel.yaml is single-source, python2/python3 compatible
2018-08-20Even better Makefile target parsing (#8819)Adam J. Stewart30-34/+301
#8223 replaced regex-based makefile target parsing with an invocation of "make -q". #8818 discovered that "make -q" can result in an error for some packages. Also, the "make -q" strategy relied on interpreting the error code, which only worked for GNU Make and not BSD Make (which was deemed acceptable at the time). As an added bonus, this implementation ignores the exit code and instead parses STDERR for any indications that the target does not exist; this works for both GNU Make and BSD Make. #8223 also updated ninja target detection to use "ninja -t targets". This does not change that behavior but makes it more-explicit with "ninja -t targets all" This also adds tests for detection of "make" and "ninja" targets.
2018-08-17install_tree: symlink handling and add 'ignore' option (#9019)scheibelp2-19/+86
Fixes #9001 #8289 added support for install_tree and copy_tree to merge into an existing directory structure. However, it did not properly handle relative symlinks and also removed support for the 'ignore' keyword. Additionally, some of the tests were overly-strict when checking the permissions on the copied files. This updates the install_tree/copy_tree methods and their tests: * copy_tree/install_tree now preserve relative link targets (if the symlink in the source directory structure is relative, the symlink created in the destination will be relative) * Added support for 'ignore' argument back to copy_tree/install_tree (removed in #8289). It is no longer the object output by shutil.ignore_patterns: you pass a function that accepts a path relative to the source and returns whether that path should be copied. * The openfoam packages (currently the only ones making use of the 'ignore' argument) are updated for the new API * When a symlink target is absolute, copy_tree and install_tree now rewrite the source prefix to be the destination prefix * copy_tree tests no longer check permissions: copy_tree doesn't enforce anything about permissions so its tests don't check for that * install_tree tests no longer check for exact permission matching since it can add file permissions
2018-08-16imports: spack uses importlib instead of imp when availableTodd Gamblin7-39/+199
- `imp` is deprecated and seems to have started having some weird issues on certain Linux versions. - In particular, the file argument to `load_source` is ignored on arch linux with Python 3.7. - `imp` is the only way to do imports in 2.6, so we'll keep it around for now and use it if importlib won't work. - `importlib` is the new import system, and it allows us to get lower-level access to the import implementation. - This consolidates all import logic into `spack.util.imp`, and make it use `importlib` if it's avialable.
2018-08-16Fix spack versions behavior when no URL (#8967)Adam J. Stewart3-12/+45
* Fix spack versions behavior when no URL * Unit test packages without URLs or safe versions
2018-08-15install_tree, copy_tree can install into existing directory structures (#8289)Adam J. Stewart3-73/+294
Replace use of `shutil.copytree` with `copy_tree` and `install_tree` functions in `llnl.util.filesystem`. - `copy_tree` copies without setting permissions. It should be used to copy files around in the build directory. - `install_tree` copies files and sets permissions. It should be used to copy files into the installation directory. - `install` and `copy` are analogous single-file functions. - add more extensive tests for these functions - update packages to use these functions.
2018-08-14tests: make the dependency patching test more completeTodd Gamblin2-3/+21
- dependency patching test didn't attempt to apply patches; just to see whether they were on the spec. - it applies the patch now and verifies that that patch was applied.
2018-08-10Branch with the meson build-system (#8467)Tom Merrick9-0/+309
* Branch with the meson build-system * Fix build_environment for dual loads and add create code * Add documentation * Fixed option list * Update build_system_guess for meson * Fixed documentation errors * Added meson to build and configure and updated documentation * fix typos
2018-08-09spec: make full_hash look like dag_hash (#8911)Scott Wittenburg1-2/+7
2018-08-09bugfix: cc handles spaces in flag variables properlyTodd Gamblin2-90/+82
- cc cleanup caused a parsing regression in flag handling - We added proper quoting to array expansions, but flag variables were never actually converted to arrays. Old code relied on this. This commit: - Adds reads to convert flags to arrays. - Makes the cc test check for improper space handling to prevent future regressions.
2018-08-08cc: refactor flag adding so that it's not in reverse orderTodd Gamblin2-29/+56
- flags were prepended in reverse order to args, but this makes it hard to see what order they'll be in on the final command line. - add them in the order they'll appear to make cc easier to maintain. - simplify code for assembling the command line - fix separator used in SPACK_SYSTEM_DIRS test
2018-08-08cc: restore ccache support in the wrapper, add a regression testTodd Gamblin2-2/+38
- Add back ccache support to the wrapper. - Add a regression test to make sure ccache is working properly.
2018-08-08cc: run shellcheck linter on the cc compiler script, minor cleanupTodd Gamblin2-12/+14
- This corrects most of the issues found by shellcheck - This also uses ':' as the delimiter for SPACK_SYSTEM_DIRS, for consistency with other variables.
2018-08-08cc: don't use sed to filter system directoriesTodd Gamblin2-42/+93
- filtering using sed causes most builds to slow down quite a bit, as the compiler wrapper has to run sed many times, and *it* runs many times - do the system directory parsing directly in bash
2018-08-08refactor: clean up and fix the cc testTodd Gamblin2-414/+503
- Add tests to ensure that RPATHs are not added in cc mode, which can cause some builds to fail. - Change cc.py to use pytest style - Instead of writing out all the flags, break the flags down into variables so that it's easy to read what each test is supposed to check. This should make cc.py more maintainable in the future.
2018-08-08bugfix: cc should not add -L or -Wl,-rpath in compile-only modeTodd Gamblin1-80/+108
- Adding -L and -Wl,-rpath to compile-only command lines ("cc mode" or "-c") causes clang (if not also other compilers) to emit warnings that confuse configure systems. - Clang will print warnings about unused command-line arguments. - This fix ensures that -L and -Wl,-rpath are not added if the compile line is just building an object file with -c - This also cleans up the cc script in several places.
2018-08-08Restore cc: package search paths come before dependency paths (#4692)Gregory Becker4-158/+330
Spack currently prepends include paths, library paths, and rpaths to the compile line. This causes problems when a header or library in the package has the same name as one exported by one of its dependencies. The *dependency's* header will be preferred over the package's, which is not what most builds expect. This also breaks some of our production codes. This restores the original cc behavior (from *very* early Spack) of parsing compiler arguments out by type (`-L`, `-I`, `-Wl,-rpath`) and reconstituting the full command at the end. `<includes> <other_args> <library dirs> <rpaths>` This differs from the original behavior in one significant way, though: it *appends* the library arguments so that dependency libraries do not shadow those in the build. This is safe because semantics aren't affected by *interleaving* `-I`, `-L`, and `-Wl,-rpath` arguments with others, only with each other (so the order of two `-L` args affects the search path, but we search for all libraries on the command line using the same search path). We preserve the following: 1. Any system directory in the paths will be listed last. 2. The root package's include/library/RPATH flags come before flags of the same type for any dependency. 3. Order will be preserved within flags passed by the build (except system paths, which are moved to be last) 4. Flags for dependencies will appear between the root flags and the system flags, and the flags for any dependency will come before those for *its* dependencies (this is for completeness -- we already guarantee this in `build_environment.py`)
2018-08-07Fix performance issue when compiling. (#8828)cedricchevalier191-4/+3
* Fix performance issue when compiling. Spack was doing active wait when compiling, spoiling one core. My fix consists in not setting any timeout for select, instead of the previous 0 second. * Fix comments about select.select timeout
2018-08-06libstdc++ detection uses os.path.realpath instead of os.readlink (#8865)Paul Chelarescu1-1/+1
2018-08-06Remove default variant bug from list of known issues (#8900)Adam J. Stewart1-32/+0
2018-08-06Docstring typo: builing -> building (#8896)George Hartzell1-1/+1
2018-08-04revert cd9691de5 (#4692) while we work on a fix.Todd Gamblin4-330/+158
2018-08-03spack spec: no extra newline with --yaml; error with no specsTodd Gamblin2-1/+16
- `spack spec` now returns an error if given no specs - removed superfluous trailing newline from `spack spec --yaml` output (only one newline now)
2018-08-03Add spack arch --operating-system and --target flagsAdam J. Stewart3-6/+40
2018-08-02libdwarf: remove use of hide_files()Todd Gamblin1-13/+0
- This was a nasty workaround due to the way our compiler wrappers used to work. We don't want to have to modify our elfutils installation to install libdwarf. - Since cd9691de5, we no longer need this because the package will always come before dependencies in our include order.
2018-08-01cc: package search paths come before dependency paths (#4692)becker334-158/+330
Spack currently prepends include paths, library paths, and rpaths to the compile line. This causes problems when a header or library in the package has the same name as one exported by one of its dependencies. The *dependency's* header will be preferred over the package's, which is not what most builds expect. This also breaks some of our production codes. This restores the original cc behavior (from *very* early Spack) of parsing compiler arguments out by type (`-L`, `-I`, `-Wl,-rpath`) and reconstituting the full command at the end. `<includes> <other_args> <library dirs> <rpaths>` This differs from the original behavior in one significant way, though: it *appends* the library arguments so that dependency libraries do not shadow those in the build. This is safe because semantics aren't affected by *interleaving* `-I`, `-L`, and `-Wl,-rpath` arguments with others, only with each other (so the order fo two `-L` args affects the search path, but we search for all libraries on the command line using the same search path). We preserve the following: 1. Any system directory in the paths will be listed last. 2. The root package's include/library/RPATH flags come before flags of the same type for any dependency. 3. Order will be preserved within flags passed by the build (except system paths, which are moved to be last) 4. Flags for dependencies will appear between the root flags and the system flags, and the flags for any dependency will come before those for *its* dependencies (this is for completeness -- we already guarantee this in `build_environment.py`)
2018-08-01docs: minor text changes (#8858)Holly3-13/+13
2018-08-01lmod: fix use of custom separator in prepend_path etc. (#8737)Stephen Herbein2-0/+19
fixes #8736
2018-07-31docs: update `spack view` argument synatx (#8839)Stephen Herbein1-2/+2
The `--dependencies` and `--exclude` flags are now arguments to the `spack view` comand rather than the `spack view symlink` command.
2018-07-29spec: `Spec.tree()` merges deptypes when only covering nodes (#8821)Todd Gamblin2-8/+56
- previously, output could be confusing when deptypes were only shown for one dependent when a node had *multiple* dependents - also fix default coverage of `Spec.tree()`: it previously defaulted to cover only build and link dependencies, but this is a holdover from when those were the only types.
2018-07-27Fixed a typo in the docs (the docker image to pull was wrong) (#8814)Massimiliano Culpo1-1/+1
2018-07-25Add docs on new top-level VCS attributesAdam J. Stewart1-108/+181
2018-07-25tests: convert mirror test to pytest with free functions.Todd Gamblin1-44/+47
2018-07-25tests: add checks and tests for consistent version() argumentsTodd Gamblin3-2/+60
- Previously, Spack didn't check the arguments you put in version() directives. - So, you could do something like this, where there are arguments for a URL fetcher AND for a git fetcher: version('1.0', md5='abc123', git='https://foo.bar', commit='feda2343') - Now, we check the arguments before constructing a fetcher, to ensure that each package has *only* arguments for a single type of fetcher. - Also added `test_package_version_consistency()` to the `package_sanity` test, so that all builtin packages are required to have valid `version()` directives.
2018-07-25tests: correct name of `check_db` to `check_repo`Todd Gamblin1-7/+6
- this test is checking the package *repository*, not the database.
2018-07-25core: differentiate package-level fetch URLs by args to `version()`Todd Gamblin2-84/+186
- packagers can specify two top-level fetch URLs if one is `url` - e.g., `url` and `git` or `url` and `svn` - allow only one VCS fetcher so we can differentiate between URL and VCS. - also clean up fetcher logic and class structure
2018-07-25core: add check for conflicting top-level fetch attributes in packagesTodd Gamblin4-6/+35
- ensure that packages can't have more than one of git, hg, svn, or url
2018-07-25style: clean up exception definitions in package.pyTodd Gamblin1-7/+2