summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2019-09-02docs: add docs for `spack find --format` and `spack find --json`Todd Gamblin3-9/+119
- add docs for `spack find --format` - add docs for `spack find --json` - update references to Spec.format() docs and add links
2019-09-02command: add `spack find --json`Todd Gamblin4-4/+52
This is another machine-readable version of `spack find`. Supplying the `--json` argument causes specs to be written out as json records, easily filered with tools like jq. e.g.: $ spack find --json python | jq -C ".[] | { name, version } " [ { "name": "python", "version": "2.7.16" }, { "name": "bzip2", "version": "1.0.8" } ]
2019-09-02command: add `spack find --format`Todd Gamblin2-23/+53
- spack find --format allows you to supply a format string and have specs output in a more machine-readable way, without dedcoration e.g.: spack find --format "{name}-{version}-{hash}" autoconf-2.69-icynozk7ti6h4ezzgonqe6jgw5f3ulx4 automake-1.16.1-o5v3tc77kesgonxjbmeqlwfmb5qzj7zy bzip2-1.0.6-syohzw57v2jfag5du2x4bowziw3m5p67 ... or: spack find --format "{hash}" icynozk7ti6h4ezzgonqe6jgw5f3ulx4 o5v3tc77kesgonxjbmeqlwfmb5qzj7zy syohzw57v2jfag5du2x4bowziw3m5p67 ... This is intended to make it much easier to script with `spack find`
2019-08-30Archive all patches in a package after install (#12653)Peter Scheibel1-1/+4
When Spack installs a package it writes the package.py file and patches to a separate repository (which reflects the state of the package at the time it was installed). Previously, Spack only wrote patches that were used at installation time. This updates the archiving step to include all patch files that are relevant to the package (in case that repository is used in another context).
2019-08-30Fujitsu compiler: Add flags for c11/c99 (#12622)t-karatsu2-0/+10
* Fujitsu compiler: Add flags for c11/c99 * Add unit tests about supported flags
2019-08-29fix packaging_guide variant table (#12296)Greg Becker1-1/+6
2019-08-27AutotoolsPackage: remove redundant libtoolize callGeoffrey M Oxberry1-4/+0
This commit removes redundant calls to `libtoolize` and `aclocal`. Some configurations, such as a Spack user using macOS with a Homebrew-installed `libtool` added to their `packages.yaml`, have `autoreconf` and GNU libtoolize installed as `glibtoolize`, but not `libtoolize`. While Spack installations of `libtool` built from source would install `glibtoolize` and symlink `libtoolize` to `glibtoolize`, an external installation of GNU libtoolize as `glibtoolize` will not have such a symlink, and thus the call `m.libtoolize()` will throw an error because `libtoolize` does not exist at the path referenced by `m.libtoolize()` (i.e., `self.spec['libtool'].prefix.bin.join('libtoolize')). However, on these same systems, `autoreconf` runs correctly, and calls `glibtoolize` instead of `libtoolize`, when appropriate. Thus, removing the call to `libtoolize` should resolve the error mentioned above. The redundant call to `aclocal` is also removed in this commit because the maintainers of GNU Automake state that "`aclocal` is expected to disappear" and suggest that downstream users never call `aclocal` directly -- rather, they suggest calling `autoreconf` instead.
2019-08-27Configuration schema are checked against jsonschema draft 4 meta-schema (#10247)Massimiliano Culpo3-3/+197
fixes #10246
2019-08-25RPackage docs: use cloud.r-project.org (#12471)Adam J. Stewart1-11/+31
2019-08-24Add implicit rpaths to compiler detection (#7153)Ben Boeckel21-8/+384
Uses code from CMake to detect implicit link paths from compilers System paths are filtered out of implicit link paths Implicit link paths added to compiler config and object under `implicit_rpaths` Implicit link paths added as rpaths to compile line through env/cc wrapper Authored by: "Ben Boeckel <ben.boeckel@kitware.com>" Co-authored by: "Peter Scheibel <scheibel1@llnl.gov>" Co-authored by: "Gregory Becker <becker33@llnl.gov>"
2019-08-23bugfix: template create constructor args (#12536)Tamara Dahlgren1-10/+10
c9e214f updated template creation by passing **kwargs to package template classes but the template classes were not updated to accept them; this adds **kwargs to package template initializers where they are needed.
2019-08-23Skip invisible non-packages in package directory. (#12467)Seth R. Johnson2-5/+11
Having a non-directory invisible file causes `spack find` to die. This fixes the logic to ignore invalid module names but only warn if they're visible. ``` NotADirectoryError: [Errno 20] Not a directory: '/spack/var/spack/repos/builtin/packages/.DS_Store/package.py' ```
2019-08-22Spack BundlePackage: a group of other packages (#11981)Tamara Dahlgren16-175/+475
This adds a special package type to Spack which is used to aggregate a set of packages that a user might commonly install together; it does not include any source code itself and does not require a download URL like other Spack packages. It may include an 'install' method to generate scripts, and Spack will run post-install hooks (including module generation). * Add new BundlePackage type * Update the Xsdk package to be a BundlePackage and remove the 'install' method (previously it had a noop install method) * "spack create --template" now takes "bundle" as an option * Rename cmd_create_repo fixture to "mock_test_repo" and relocate it to shared pytest fixtures * Add unit tests for BundlePackage behavior
2019-08-22compiler install: fix bug in construction of compiler spec (#12507)Greg Becker1-1/+2
2019-08-21'spack spec --yaml' command now writes build deps to yaml (#12317)Peter Scheibel1-1/+2
This allows "spack spec --yaml" to generate a spec YAML file that can be used with "spack install -f". Before, this would fail in cases where the spec had build dependencies.
2019-08-21Emacs directives for fill-column setting.Chris Green1-0/+5
2019-08-21Fetch strategies: new global option `no_cache`, new git option `get_full_repo`.Chris Green4-35/+113
* All fetch strategies now accept the Boolean version keyword option `no_cache` in order to allow per-version control of cache-ability. * New git-specific version keyword option `get_full_repo` (Boolean). When true, disables the default `--depth 1` and `--single-branch` optimizations that are applied if supported by the git version and (in the former case) transport protocol. * The try / catch blog attempting `--depth 1` and retrying on failure has been removed in favor of more accurately ascertaining when the `--depth` option should work based on git version and protocol choice. Any failure is now treated as a real problem, and the clone is only attempted once. * Test improvements: * `mock_git_repository.checks[type_of_test].args['git']` is now specified as the URL (with leading `file://`) in order to avoid complaints when using `--depth`. * New type_of_test `tag-branch`. * mock_git_repository now provides `git_exe`. * Improved the action of the `git_version` fixture, which was previously hard-wired. * New tests of `--single-branch` and `--depth 1` behavior. * Add documentation of new options to the packaging guide.
2019-08-21Fix to option text for help command. (#12444)Chris Green2-3/+3
2019-08-21properly format message with all paths that could not be used. also report ↵Peter Scheibel1-3/+4
canonicalized paths vs. Spack config variables like 'tempdir' (#12492)
2019-08-21pkg command uses a working directory for git invocations rather than -C. ↵Chris Green1-5/+23
(#12491)
2019-08-20permissions: fix file permissions on intermediate install directories (#12399)Greg Becker2-30/+64
- mkdirp now takes arguments to allow it to properly set permissions on created directories. - Two arguments (group and mode) set permissions for the leaf directory. - Intermediate directories can inherit permissions from either the topmost existing directory (the parent) or the leaf.
2019-08-20stage: avoid using group name in _first_accessible_path()Todd Gamblin1-3/+2
On machines where $TMP is owned by a gid with no name, this avoids the following error when the default spack stage does not exist: (spackbook):spack$ spack clean ==> Removing all temporary build stages ==> Error: 'getgrgid(): gid not found: 57095' Spack needs to deal with gids directly unless users pass them in.
2019-08-19cleanup: remove extraneous prints (#12474)Tamara Dahlgren1-2/+0
Removes messages accidentally introduced in #12072 .
2019-08-19features: Update compiler caching (#7675)Peter Scheibel2-34/+60
Compiler caching was using the `id()` function to refer to configuration dictionary objects. If these objects are garbage-collected, this can produce incorrect results (false positive cache hits). This change replaces `id()` with an object that keeps a reference to the config dictionary so that it is not garbage-collected.
2019-08-19features: Remove stage symlinks (#12072)Tamara Dahlgren16-307/+399
Fixes #11163 The goal of this work is to simplify stage directory structures by eliminating use of symbolic links. This means, among other things, that` $spack/var/spack/stage` will no longer be the core staging directory. Instead, the first accessible `config:build_stage` path will be used. Spack will no longer automatically append `spack-stage` (or the like) to configured build stage directories so the onus of distinguishing the directory from other work -- so the other work is not automatically removed with a `spack clean` operation -- falls on the user.
2019-08-16Do not report on packages installed from the cache (#12336)Zack Galbreath3-1/+15
Skip generating reports for any packages that were found in the binary cache.
2019-08-15mirrors: make `spack mirror` environment aware (#12345)Greg Becker3-4/+38
Allow "spack mirror" with no spec arguments to create a mirror for the active environment (download all concretized specs in the environment)
2019-08-13Fix "spack loads" when retrieving upstream modules (#12203)Peter Scheibel1-9/+3
Fixes #12062 406c791 addressed "spack module load" for upstream modules but not the "spack module loads" command. This applies the same fixes from 406c791 to "spack module loads".
2019-08-11bugfix: Python 2.6 parsing error (#11867)Massimiliano Culpo5-12/+4
Apparently shlex.split can't deal with unicode encoded characters in Python2.6. The solution is to convert to str before calling the function.
2019-08-08bugfix: nested matrices in spec lists (#12320)Greg Becker6-27/+88
* stack concretization: fix handling of variant names with dashes * spec_list: bugfix for handling nested matrices
2019-08-08Allow Spack to uninstall external extensions without permissions (#11991)Adam J. Stewart1-3/+7
* Allow Spack to uninstall external extensions without permissions Raise when self.projections is not empty
2019-08-08docs: fix compiler config (#12330)Pariksheet Nanda1-2/+3
It's no longer possible to set compiler flags under as an entry under "paths" in compilers.yaml; instead the user must list these under the "flags" section. This updates the docs accordingly.
2019-08-08Fix typo in yaml config docs (#12312)Adam J. Stewart1-1/+1
2019-08-07bugfix: spack stacks mixed dependency and non-dep constraints (#12315)Greg Becker1-1/+1
Spack stacks drop invalid dependencies applied to packages by a spec_list matrix operation Without this fix, Spack would raise an error if orthogonal dependency constraints and non-dependency constraints were applied to the same package by a matrix and the dependency constraint was invalid for that package. This is an error, fixed by this PR. An example failing configuration: spack: definitions: - packages: [libelf, hdf5+mpi] - compilers: ['%gcc'] - mpis: [^openmpi] specs: - matrix: - $packages - $compilers - $mpis
2019-08-07compiler bootstrapping: enable both single-node and multi-node DAGS(#12310)Peter Scheibel1-7/+15
5f74f22 enabled installing compilers for dependencies but not for the root package (and in particular not for DAGs which consist of one package) this enables bootstrapping compilers for both types of DAGs
2019-08-06tests: explain and test dependency flattening routines (#11993)Tamara Dahlgren2-1/+44
- Add comments to explain that `install_dependency_symlinks` and `flatten_dependencies` are actually used. - Add a test that exercises the routines.
2019-08-06mirrors: mirror config should use spack variable expansions (#9027)Greg Becker2-9/+17
- ensure that `$spack` and other variables are substituted into mirror paths
2019-08-06stacks: fix reference handling in env.write() (#12096)Greg Becker2-2/+27
* stacks: Fix env.write to properly write references * stacks: regression test for 12095
2019-08-03Fix typo in config (#12267)Pariksheet Nanda1-1/+1
Using "compilers" with the "s" is an invalid config section and throws an error. Traceback (most recent call last): File "spack/bin/spack", line 48, in <module> sys.exit(spack.main.main()) File "/home/omsai/src/libkmap/spack/lib/spack/spack/main.py", line 633, in main env = ev.find_environment(args) File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 263, in find_environment return Environment(env) File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 534, in __init__ self._read_manifest(f) File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 561, in _read_manifest self.yaml = _read_yaml(f) File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 402, in _read_yaml validate(data, filename) File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 395, in validate e, data, filename, e.instance.lc.line + 1) spack.config.ConfigFormatError: /home/omsai/src/libkmap/spack.yaml:15: Additional properties are not allowed ('compilers' was unexpected)
2019-08-03Fix CNL version detection (#12207)Adam J. Stewart2-6/+76
2019-08-02Refactor UI logic out of Environment.concretize (#12213)Massimiliano Culpo4-15/+34
Environment.concretize returns newly-concretized specs rather than printing them; as a result, the _display argument is removed from Environment.concretize (originally only used to avoid printing specs during unit testing). Command logic which invokes Environment.concretize prints these explicitly.
2019-08-01Fix QT4 build for mac (#10944)Seth R. Johnson1-2/+2
This updates the Spack QT package to enable building qt version 4 on MacOS. This includes the following changes to the qt package: * add version 4.8.7 * add option to build with or without shared libs * add options to disable tools, ssl, sql, and freetype support * add qt4-tools patch when building qt@4+tools * add option to build as a framework (only available on MacOS) * replace qt4-el-capitan patch with qt4-mac patch (which includes the edits from qt4-el-capitan) * apply qt4-pcre-include-conflict.patch only for version 4.8.6 (rather than all 4.x versions) * apply qt4-gcc-and-webkit.patch for 4.x versions before 4.8.7 and create a separate qt4-gcc-and-webkit-487.patch for version 4.8.7 * update patch function for qt@4 on MacOS to update configure variables relevant to Spack (e.g. PREFIX) * add option to build freetype with Spack, as a vendored dependency of QT, or not at all (default is to build with Spack) This includes the following edits outside of the qt package: * Update MacOS version utility function to return all parts of the Mac version (rather than just the first two) * gettext package: implement "libs" * python package: add gettext as a dependency
2019-08-01Buildcache: skip binary string replacement with padding when the new install ↵Patrick Gartung1-5/+32
path is longer than the old install path. (#12227) * Raise an exception and exit with a meaningful message when binary path substitution fails. * Skip binary text replacement with padding and issue a warning when the new install path is longer than the old install path.
2019-07-31should not need bin prefix on spack command (#12215)Levi Baber1-1/+1
2019-07-31docs: update tutorial slides for PEARC19Todd Gamblin1-9/+9
2019-07-31Update Package Creation Tutorial for PEARC19 (#12209)Adam J. Stewart7-233/+235
* Update Package Creation Tutorial for PEARC19 * A few remaining fixes * Fix section link
2019-07-31CMake: make ninja verbose as well (#12200)Christoph Junghans1-0/+1
2019-07-31Print environment name when removing active env (#12198)Javier Cervantes1-1/+2
2019-07-31Fix Sphinx deprecation warning when building docs (#12205)Adam J. Stewart1-3/+2
* Fix Sphinx deprecation warning when building docs * Set minimum required version of Sphinx
2019-07-30Add SIPPackage base class (#12157)Adam J. Stewart8-0/+279