summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2019-09-18Update buildcache creation and installation to allow mach-o binary ↵Patrick Gartung3-138/+200
relocation using py-machotools on linux or macos. (#12858) Update py-machotools dependencies and versions.
2019-09-17Support yaml paths anywhere specs are handled on CLI (#12561)Scott Wittenburg4-19/+318
Update command-line (CLI) parsing to understand references to yaml files that store Spack specs. Where a file reference is encountered, the full Spec in the file will be read in. A file reference may appear anywhere that a spec could appear before. For example, if you write "spack spec -y openmpi > openmpi.yaml" you may then install the spec using the yaml file by running "spack install ./openmpi.yaml"; you can also refer to dependencies in this way (e.g. "spack install foo^./openmpi.yaml"). There are two requirements for file references: * A file path entered on the CLI must include a "/" even if the file exists in your current working directory. For example, if you create an openmpi.yaml file as above and run "spack install openmpi.yaml" from the same directory, it will report an error. * A file path entered on the CLI must end with ".yaml" This commit adds error messages to clearly inform the user of both violations.
2019-09-17implicit rpaths filtering (#12789)Peter Scheibel14-52/+191
* implicit_rpaths are now removed from compilers.yaml config and are always instantiated dynamically, this occurs one time in the build_environment module * per-compiler list required libraries (e.g. libstdc++, libgfortran) and whitelist directories from rpaths including those libraries. Remove non-whitelisted implicit rpaths. Some libraries default for all compilers. * reintroduce 'implicit_rpaths' as a config variable that can be used to disable Spack insertion of compiler RPATHs generated at build time.
2019-09-17Fix generic body during package creation (#12804)Adam J. Stewart1-14/+14
Fixes incomplete change in #11981 Use the proper variable (`body_def`) during package creation for package subclasses.
2019-09-16Allow build to proceed with failed compiler lib detection (#12756)Adam J. Stewart1-0/+4
Fixes #12732 Fixes #12767 c22a145 added automatic detection and RPATHing of compiler libraries to Spack builds. However, in cases where the parsing/detection logic fails this was terminating the build. This makes the compiler library detection "best-effort" and reports an issue when the detection fails rather than terminating the build.
2019-09-16Quote shell variable values (#12759)Adam J. Stewart1-2/+2
This updates logic which sets shell variables to quote the values, which is necessary when the value contains a space (e.g. PATH).
2019-09-13Fixes identified in ecp facilities hackathon fixes:Scott Wittenburg2-3/+2
- improve mirror git repo management - minio s3 implementation needs endpoint_url - Remove image key from rebuild-index job - Remove image, rely on tags instead
2019-09-13Implement an optional compiler bootstrapping phaseScott Wittenburg7-251/+515
2019-09-13Remove CombinatorialSpecSet in favor of environments + stacksScott Wittenburg9-705/+83
2019-09-13Refactor release-jobs cmd based on use of environments (no docker either)Scott Wittenburg2-291/+173
2019-09-13Add schema for new gitlab-ci and cdash additions to environmentScott Wittenburg4-0/+220
2019-09-11module parsing: make heuristic to get paths from modules more robust (#12693)Greg Becker2-36/+87
* module parsing: make heuristic to get paths from modules more robust * refactor module parsing
2019-09-09clang: add another regex for version detection (#12769)Massimiliano Culpo2-0/+5
This is similar to #10191. The Ubuntu package for clang 8.0.0 displays a very unusual version string, and we need this new regex to detect it as just 8.0.0 Unit test have been complemented by the output that was failing detection.
2019-09-08Fix PyQt installation (#12479)Adam J. Stewart2-15/+41
* Fix PyQt installation * Switch dependency type * Replace SIP dependency with resource * Relax py-pyqt4 Qt dependency
2019-09-07flake8: make entire repository pass `spack flake8 -a`Todd Gamblin8-45/+46
- Fix trailing whitespace missed by the bug described in #12755. - Fix other style issues that have crept in over time (this can happen when flake8 adds new checks with new versions)
2019-09-07flake8: only add E501 exemptions when absolutely necessary (#12755)Todd Gamblin1-2/+13
E501 (line too long) exemptions are probably our most common ones -- we add them for directives, URLs, hashes, etc. in packages. But we currently add them even when a line *doesn't* need them, which can mask trailing whitespace errors. This changes `spack flake8` so that it will only add E501 exemptions if the line is *actually* too long. Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
2019-09-07Update docs on module loads command (#12702)Adam J. Stewart1-2/+2
2019-09-05make env view loading unconditional (#12719)Greg Becker1-4/+2
2019-09-04Improve mock_archive versatility; Remove unwanted ALLOWED_ARCHIVE_TYPES. ↵Chris Green3-7/+15
(#12513) mock_archive can now take multiple extension / tar option pairs (default matches old behavior). url_fetch.test_fetch tests more archive types. compression.EXTS split into EXTS and NOTAR_EXTS to avoid unwanted, non-meaningful combinatoric extensions such as .tar.tbz2.
2019-09-04specs: better lookup by hash; allow references to missing dependency hashesTodd Gamblin3-16/+155
- previously spec parsing didn't allow you to look up missing (but still known) specs by hash - This allows you to reference and potentially reinstall force-uninstalled dependencies - add testing for force uninstall and for reference by spec - cmd/install tests now use mutable_database
2019-09-03Use spack/user-specific stage root by default; stage cleaning (#12516)Tamara Dahlgren7-72/+166
* When cleaning the stage root, only remove directories that appear to be used for staging Spack packages. Previously Spack was clearing all directories in the stage root, which could remove content not related to Spack if the user chose a staging root which contains files/directories not managed by Spack. * The documentation is updated with warnings about choosing a stage directory that is only managed by Spack (although generally the check added in this PR for "spack clean" should avoid removing content that was not created by Spack) * The default stage directory (in config.yaml) is now $tempdir/$user/spack-stage and the logic is updated to omit the $user portion of this path if $tempdir already contains a $user directory. * When creating stage root assign user read/write permissions to all directories in the path under $user. Previously Spack was assigning the permissions of the first existing parent directory
2019-09-03perf: spack find -p now does only one DB transactionTodd Gamblin1-9/+12
`spec.prefix` reads from Spack's database, and if you do this with multiple consecutive read transactions, it can take a long time. Or, at least, you can see the paths get written out one by one. This uses an outer read transaction to ensure that actual disk locks are acquired only once for the whole `spack find` operation, and that each transaction inside `spec.prefix` is an in-memory operation. This speeds up `spack find -p` a lot.
2019-09-02tests: add tests for `spack extensions` commandTodd Gamblin3-68/+128
- add tests for `spack extensions` - refactor `test_activations` test to use real extensions.
2019-09-02tests and completions for `spack find --json` and `spack find --format`Todd Gamblin4-17/+214
2019-09-02tests: remove extraneous print statementTodd Gamblin1-4/+0
2019-09-02refactor: clean up `spack find`, make `spack find -dp` work properlyTodd Gamblin4-153/+132
Refactor `spack.cmd.display_specs()` and `spack find` so that any options can be used together with -d. This cleans up the display logic considerably, as there are no longer multiple "modes".
2019-09-02docs: add docs for `spack find --format` and `spack find --json`Todd Gamblin3-9/+119
- add docs for `spack find --format` - add docs for `spack find --json` - update references to Spec.format() docs and add links
2019-09-02command: add `spack find --json`Todd Gamblin4-4/+52
This is another machine-readable version of `spack find`. Supplying the `--json` argument causes specs to be written out as json records, easily filered with tools like jq. e.g.: $ spack find --json python | jq -C ".[] | { name, version } " [ { "name": "python", "version": "2.7.16" }, { "name": "bzip2", "version": "1.0.8" } ]
2019-09-02command: add `spack find --format`Todd Gamblin2-23/+53
- spack find --format allows you to supply a format string and have specs output in a more machine-readable way, without dedcoration e.g.: spack find --format "{name}-{version}-{hash}" autoconf-2.69-icynozk7ti6h4ezzgonqe6jgw5f3ulx4 automake-1.16.1-o5v3tc77kesgonxjbmeqlwfmb5qzj7zy bzip2-1.0.6-syohzw57v2jfag5du2x4bowziw3m5p67 ... or: spack find --format "{hash}" icynozk7ti6h4ezzgonqe6jgw5f3ulx4 o5v3tc77kesgonxjbmeqlwfmb5qzj7zy syohzw57v2jfag5du2x4bowziw3m5p67 ... This is intended to make it much easier to script with `spack find`
2019-08-30Archive all patches in a package after install (#12653)Peter Scheibel1-1/+4
When Spack installs a package it writes the package.py file and patches to a separate repository (which reflects the state of the package at the time it was installed). Previously, Spack only wrote patches that were used at installation time. This updates the archiving step to include all patch files that are relevant to the package (in case that repository is used in another context).
2019-08-30Fujitsu compiler: Add flags for c11/c99 (#12622)t-karatsu2-0/+10
* Fujitsu compiler: Add flags for c11/c99 * Add unit tests about supported flags
2019-08-29fix packaging_guide variant table (#12296)Greg Becker1-1/+6
2019-08-27AutotoolsPackage: remove redundant libtoolize callGeoffrey M Oxberry1-4/+0
This commit removes redundant calls to `libtoolize` and `aclocal`. Some configurations, such as a Spack user using macOS with a Homebrew-installed `libtool` added to their `packages.yaml`, have `autoreconf` and GNU libtoolize installed as `glibtoolize`, but not `libtoolize`. While Spack installations of `libtool` built from source would install `glibtoolize` and symlink `libtoolize` to `glibtoolize`, an external installation of GNU libtoolize as `glibtoolize` will not have such a symlink, and thus the call `m.libtoolize()` will throw an error because `libtoolize` does not exist at the path referenced by `m.libtoolize()` (i.e., `self.spec['libtool'].prefix.bin.join('libtoolize')). However, on these same systems, `autoreconf` runs correctly, and calls `glibtoolize` instead of `libtoolize`, when appropriate. Thus, removing the call to `libtoolize` should resolve the error mentioned above. The redundant call to `aclocal` is also removed in this commit because the maintainers of GNU Automake state that "`aclocal` is expected to disappear" and suggest that downstream users never call `aclocal` directly -- rather, they suggest calling `autoreconf` instead.
2019-08-27Configuration schema are checked against jsonschema draft 4 meta-schema (#10247)Massimiliano Culpo3-3/+197
fixes #10246
2019-08-25RPackage docs: use cloud.r-project.org (#12471)Adam J. Stewart1-11/+31
2019-08-24Add implicit rpaths to compiler detection (#7153)Ben Boeckel21-8/+384
Uses code from CMake to detect implicit link paths from compilers System paths are filtered out of implicit link paths Implicit link paths added to compiler config and object under `implicit_rpaths` Implicit link paths added as rpaths to compile line through env/cc wrapper Authored by: "Ben Boeckel <ben.boeckel@kitware.com>" Co-authored by: "Peter Scheibel <scheibel1@llnl.gov>" Co-authored by: "Gregory Becker <becker33@llnl.gov>"
2019-08-23bugfix: template create constructor args (#12536)Tamara Dahlgren1-10/+10
c9e214f updated template creation by passing **kwargs to package template classes but the template classes were not updated to accept them; this adds **kwargs to package template initializers where they are needed.
2019-08-23Skip invisible non-packages in package directory. (#12467)Seth R. Johnson2-5/+11
Having a non-directory invisible file causes `spack find` to die. This fixes the logic to ignore invalid module names but only warn if they're visible. ``` NotADirectoryError: [Errno 20] Not a directory: '/spack/var/spack/repos/builtin/packages/.DS_Store/package.py' ```
2019-08-22Spack BundlePackage: a group of other packages (#11981)Tamara Dahlgren16-175/+475
This adds a special package type to Spack which is used to aggregate a set of packages that a user might commonly install together; it does not include any source code itself and does not require a download URL like other Spack packages. It may include an 'install' method to generate scripts, and Spack will run post-install hooks (including module generation). * Add new BundlePackage type * Update the Xsdk package to be a BundlePackage and remove the 'install' method (previously it had a noop install method) * "spack create --template" now takes "bundle" as an option * Rename cmd_create_repo fixture to "mock_test_repo" and relocate it to shared pytest fixtures * Add unit tests for BundlePackage behavior
2019-08-22compiler install: fix bug in construction of compiler spec (#12507)Greg Becker1-1/+2
2019-08-21'spack spec --yaml' command now writes build deps to yaml (#12317)Peter Scheibel1-1/+2
This allows "spack spec --yaml" to generate a spec YAML file that can be used with "spack install -f". Before, this would fail in cases where the spec had build dependencies.
2019-08-21Emacs directives for fill-column setting.Chris Green1-0/+5
2019-08-21Fetch strategies: new global option `no_cache`, new git option `get_full_repo`.Chris Green4-35/+113
* All fetch strategies now accept the Boolean version keyword option `no_cache` in order to allow per-version control of cache-ability. * New git-specific version keyword option `get_full_repo` (Boolean). When true, disables the default `--depth 1` and `--single-branch` optimizations that are applied if supported by the git version and (in the former case) transport protocol. * The try / catch blog attempting `--depth 1` and retrying on failure has been removed in favor of more accurately ascertaining when the `--depth` option should work based on git version and protocol choice. Any failure is now treated as a real problem, and the clone is only attempted once. * Test improvements: * `mock_git_repository.checks[type_of_test].args['git']` is now specified as the URL (with leading `file://`) in order to avoid complaints when using `--depth`. * New type_of_test `tag-branch`. * mock_git_repository now provides `git_exe`. * Improved the action of the `git_version` fixture, which was previously hard-wired. * New tests of `--single-branch` and `--depth 1` behavior. * Add documentation of new options to the packaging guide.
2019-08-21Fix to option text for help command. (#12444)Chris Green2-3/+3
2019-08-21properly format message with all paths that could not be used. also report ↵Peter Scheibel1-3/+4
canonicalized paths vs. Spack config variables like 'tempdir' (#12492)
2019-08-21pkg command uses a working directory for git invocations rather than -C. ↵Chris Green1-5/+23
(#12491)
2019-08-20permissions: fix file permissions on intermediate install directories (#12399)Greg Becker2-30/+64
- mkdirp now takes arguments to allow it to properly set permissions on created directories. - Two arguments (group and mode) set permissions for the leaf directory. - Intermediate directories can inherit permissions from either the topmost existing directory (the parent) or the leaf.
2019-08-20stage: avoid using group name in _first_accessible_path()Todd Gamblin1-3/+2
On machines where $TMP is owned by a gid with no name, this avoids the following error when the default spack stage does not exist: (spackbook):spack$ spack clean ==> Removing all temporary build stages ==> Error: 'getgrgid(): gid not found: 57095' Spack needs to deal with gids directly unless users pass them in.
2019-08-19cleanup: remove extraneous prints (#12474)Tamara Dahlgren1-2/+0
Removes messages accidentally introduced in #12072 .
2019-08-19features: Update compiler caching (#7675)Peter Scheibel2-34/+60
Compiler caching was using the `id()` function to refer to configuration dictionary objects. If these objects are garbage-collected, this can produce incorrect results (false positive cache hits). This change replaces `id()` with an object that keeps a reference to the config dictionary so that it is not garbage-collected.