summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2019-02-12Enhancement to module autoload documentation (#10310)George Hartzell1-1/+15
* Note that `none` is the default for lmod autoload Save a bit of confusion by *explicitly* pointing out that `none` is the default value for autoload in the lmod module file generator. * Add a tip re building software externally Add a tip about using `autoload: all` when building packages outside of the tree that use artifacts (e.g. libraries, includes) within the tree.
2019-02-12Allow secondary generators when building with CMake. (#9324)Chris Green2-4/+50
CMake supports the notion of secondary generators which provide extra information to (e.g.) IDEs over and above that normally provided by the primary generator. Spack only supports the 'Unix Makefiles' and 'Ninja' primary generators but was not parsing out the primary generator when a secondary generator was also included (e.g. for a generator attribute like 'Codeblocks - Ninja'). This adds a regex for extracting the primary generator for validation. Since the secondary generator is irrelevant to a Spack build, it is passed on to CMake without further validation.
2019-02-08CudaPackage: fix wrong version range (#10551)Denis Davydov1-1/+1
2019-02-07log-parse: fix error message when no error lines are found (#10543)Massimiliano Culpo1-1/+1
2019-02-06Add x86_64 as target to cray platform (#10369)Mario Melara1-0/+4
Add x86_64 as a target for the Cray platform, and also designate it as the default front_end target.
2019-02-06CUDA compiler conflicts for Linux (#10460)Nichols A. Romero1-14/+69
* CUDA compiler conflicts for Linux. * Add Volta and Turing GPUs. * Add mandatory conflict for Volta and Turing GPUs. * Revert "CUDA compiler conflicts for Linux." This reverts commit 7d4ff654ac53aad272c59e9f7f8bb3fbb32bcec4. * Compiler conflicts introduced from previous commit into CUDA packaged moved and integrated into CUDA build system. * More conversative with compiler conflicts for cuda 10.0.130, since I don't know what will happen with future cuda 10.x releases. * Correct off-by-one errors in clang conflicts for x86_64 Linux. * No restrictions on Apple Clang compiler until we are able to distinguish Xcode clang from github clang more easily. Note to fix this in the future. * Change comment to clarify that github clang refers to LLVM clang. * Fix and simplify index range. * Fix overlapping conflicts for CUDA 10.0.130 * Removed extra ^cuda from conflict.
2019-01-29debug output includes modulecmd output (#9476)Peter Scheibel1-7/+28
Debug output now includes the output of modulecmd executions. Only output module content when a failure occurs; always report when a module is loaded/unloaded.
2019-01-24Environments: fix bug for install of external packages (#10437)Peter Scheibel1-0/+1
"spack install" will install all packages added to the current environment. When this included external packages, the environment update would fail because it would attempt to copy log files that were only generated if Spack handled the install itself. This skips that step for external packages.
2019-01-24Allow `spack install --overwrite` for nonexistent or multiple packages (#9201)Greg Becker2-25/+78
* Allow overwrite nonexistent and multiple packages initial implementation give one prompt to users instead of a prompt per spec testing * flake * bugfix: install overwrite check each spec against installed * python3 compliance for filter/map
2019-01-14tutorial: update tutorial materials for ECP19Todd Gamblin1-14/+12
2019-01-13version bump: v0.12.1Gregory Becker1-1/+1
2019-01-12cc: clean up cray compilers, fix issues with case-insensitive filesystems ↵Adam J. Stewart9-9/+6
(#10323) * Remove Cray CC compilers causing problems on case-insensitive filesystems * cray -> cce * Ensure that compiler-specific directory comes first in build-env * Point to compiler-specific symlinks
2019-01-11Binary caching bugfix: symlink relocation (#10073)Greg Becker3-9/+131
Binary caches of packages with absolute symlinks had broken symlinks. As a stopgap measure, #9747 addressed this by replacing symlinks with copies of files when creating binary cached packages. This reverts #9747 and instead, either relative-izes the symlink or rewrites the target. If the binary cache is created using '--rel' (as in "spack buildcache create --rel...") then absolute symlinks will be replaced with relative symlinks (in addition to making RPATHs relative as before); otherwise they are rewritten (when the binary cache is unpacked and installed).
2019-01-11enh: `buildcache list` should behave similar to `find` (#10052)Matthias Wolf1-15/+10
The current output of buildcache list is very verbose and I feel like some details are getting lost. By making the output similar to find, I think users will be able to get a better overview of what is stored in the cache.
2019-01-11dealii: fix concretization of xsdk package (#10288)Denis Davydov1-0/+11
* dealii: fix concretization of xsdk package * tests: add concretization tests for deal.II and xSDK, which are often broken due to limitations in the concretizer * use pytest.mark.parametrize
2019-01-10spack versions: only list safe versions (#10004)Adam J. Stewart2-10/+27
* spack versions: only list safe versions * Add unit tests for spack versions -s
2019-01-09Allow combinatorial projections in views (#9679)Greg Becker14-147/+500
Allow customizing views with Spec-formatted directory structure Allow views to specify projections that are more complicated than merging every package into a single shared prefix. This will allow sites to configure a view for the way they want to present packages to their users; for example this can be used to create a prefix for each package but omit the DAG hash from the path. This includes a new YAML format file for specifying the simplified prefix for a spec in a view. This configuration allows the use of different prefix formats for different specs (i.e. specs depending on MPI can include the MPI implementation in the prefix). Documentation on usage of the view projection configuration is included. Depending on the projection configuration, paths are not guaranteed to be unique and it may not be possible to add multiple installs of a package to a view.
2019-01-09tutorial basics section: fix gcc install version (#10298)Greg Becker1-1/+1
2019-01-08bug fix: copy permissions when staging (#10285)Owen Solberg1-3/+3
Fixes #10284 #10152 replaced shutil.move with llnl's copy and copy_tree for resources. This did not copy permissions so led to later failures if an executable was copied (e.g. a configure script). This uses install/install_tree instead, which preserve permissions.
2019-01-08Armcompiler (#9840)Srinath Vadlamani5-24/+24
* Initial compiler support * added arm.py * Changed licence to Arm suggested header * Changed licence to the same as clang.py Main author of file is Nick Forrington <Nick.Forrington@arm.com> Minor changes by Srinath Vadlamani <srinath.vadlamani@arm.com> * compilers: add arm compiler detection to Spack - added arm.py with support for detecting `armclang` and `armflang` Co-authored-by: Srinath Vadlamani <srinath.vadlamani@arm.com> * Changed to using get get_compiler_version * linking to general cc for arm compiler * For arm compiler add CFLAGS to use compiler-rt rtlib. * Escape for special characters in rexep * Cleaned up for Flake8 to pass. * libcompiler-rt should be part of the LDFLAGS not CFLAGS * fixed m4 when using clang to used LDFLAGS. Fixed comments for arm.py to display compiler --version output with # NOAQ for flakes pass. * added arm compilers * proper linked names
2019-01-07For spec-file-based installs, store the initial spec read from the file as ↵Peter Josef Scheibel1-0/+1
the abstract spec
2019-01-04Multi-valued variants: better support for combinations (#9481)Massimiliano Culpo9-19/+546
This enforces conventions that allow for correct handling of multi-valued variants where specifying no value is an option, and adds convenience functionality for specifying multi-valued variants with conflicting sets of values. This also adds a notion of "feature values" for variants, which are those that are understood by the build system (e.g. those that would appear as configure options). In more detail: * Add documentation on variants to the packaging guide * Forbid usage of '' or None as a possible variant value, in particular as a default. To indicate choosing no value, the user must explicitly define an option like 'none'. Without this, multi-valued variants with default set to None were not parsable from the command line (Fixes #6314) * Add "disjoint_sets" function to support the declaration of multi-valued variants with conflicting sets of options. For example a variant "foo" with possible values "a", "b", and "c" where "c" is exclusive of the other values ("foo=a,b" and "foo=c" are valid but "foo=a,c" is not). * Add "any_combination_of" function to support the declaration of multi-valued variants where it is valid to choose none of the values. This automatically defines "none" as an option (exclusive with all other choices); this value does not appear when iterating over the variant's values, for example in "with_or_without" (which constructs autotools option strings from variant values). * The "disjoint_sets" and "any_combination_of" methods return an object which tracks the possible values. It is also possible to indicate that some of these values do not correspond to options understood by the package's build system, such that methods like "with_or_without" will not define options for those values (this occurs automatically for "none") * Add documentation for usage of new functions for specifying multi-valued variants
2019-01-04Add tests for expanding/non-expanding resourcesPeter Josef Scheibel1-3/+92
2019-01-04stage: fix resources being deleted from local cache (#10152)Michael Kuhn1-4/+9
Non-expanded resources were being deleted from the cache on account of two behaviors: * ResourceStage was moving files rather than copying them, and uses "os.path.realpath" to resolve symlinks * CacheFetchStrategy creates a symlink to a cached resource rather than copying it This alters the first behavior: ResourceStage now copies the file rather than moving it.
2019-01-03Add missing code block (#10243)Ben Zwick1-0/+2
Otherwise the `--` in `--dot` is rendered as a single en-dash.
2019-01-02patch: split up fetch and clean into separate methods (#10150)Michael Kuhn5-50/+71
"mirror create" was invoking a package's do_patch method in order to retrieve and archive URL patches. If a package implements a "patch" method, this is also called as part of do_patch; this failed when the package-specific implementation referred to environment variables that are only available at the time the package is built (e.g. "spack_cc"). This change introduces fetch and clean methods for patches. They are no-ops for FilePatch but perform the appropriate actions for UrlPatch. This allows "mirror create" to invoke do_fetch, which does not call the package's patch method.
2019-01-02Removed an extra comma after a dict literal (#10236)Massimiliano Culpo1-1/+1
The extra comma was turning a dict into a tuple, and caused errors during validation of the schema.
2019-01-01flake8: fix flake8 issues with invalid escapesTodd Gamblin6-81/+81
- in many files, regular strings were used in places where raw strings should've been used. - convert these to raw strings and get rid of new flake8 errors
2019-01-01copyright: update license headers for 2013-2019 copyright.Todd Gamblin379-383/+383
2019-01-01license: license command prints sorted, non-redundant resultsTodd Gamblin1-3/+7
- spack license command now ignores symlinks - spack license list-files now prints sorted output, and checks for files we've already seen.
2019-01-01Improve validation of modules.yaml (#9878)Massimiliano Culpo8-169/+269
This PR improves the validation of `modules.yaml` by introducing a custom validator that checks if an attribute listed in `properties` or `patternProperties` is a valid spec. This new check applied to the test case in #9857 gives: ```console $ spack install szip ==> Error: /home/mculpo/.spack/linux/modules.yaml:5: "^python@2.7@" is an invalid spec [Invalid version specifier] ``` Details: * Moved the set-up of a custom validator class to spack.schema * In Spack we use `jsonschema` to validate configuration files against a schema. We also need custom validators to enforce writing default values within "properties" or "patternProperties" attributes. * Currently, validators were customized at the place of use and with the recent introduction of environments that meant we were setting-up and using 2 different validator classes in two different modules. * This commit moves the set-up of a custom validator class in the `spack.schema` module and refactors the code in `spack.config` and `spack.environments` to use it. * Added a custom validator to check if an attribute is a valid spec * Added a custom validator that can be used on objects, which yields an error if the attribute is not a valid spec. * Updated the schema for modules.yaml * Updated modules.yaml to fix a few inconsistencies: - a few attributes were not tested properly using 'anyOf' - suffixes has been updated to also check that the attribute is a spec - hierarchical_scheme has been updated to hierarchy * Removed $ref from every schema * $ref is not composable or particularly legible * Use python dicts and regular old variables instead.
2018-12-30bugfix: nested directives were broken in python 3Todd Gamblin2-1/+25
- The nested directive implementation was broken for python 3 - directive results were not properly removed from the directive list when it was processed in the DirectiveMeta metaclass. - the issue was that remove_directives only descended into a list or tuple, but in Python3, the initial value passed to the function is a view of dictionary values. - make it a list to fix things, and add a regression test.
2018-12-30commands: add `spack resource` command to inspect downloadable filesTodd Gamblin3-0/+211
- currently just looks at patches - allows you to find out which package applied a patch to a spec - intended to work with tarballs and resources in the future. - add tab completion for `spack resource` and subcommands
2018-12-30tests: add a test to make sure that patched specs can be round-trippedTodd Gamblin1-30/+101
- previously, if a concrete sub-DAG with patched specs was written out and read back in, its patches would not be found because the dependent that patched it was no longer in the DAG. - Add a test to ensure that the PatchCache handles this case. - Also add tests to ensure that patch objects are properly created from Specs -- previously we only checked that the patches were on the Spec.
2018-12-30patches: add a per-repository patch indexTodd Gamblin8-116/+339
- this fixes a bug where if we save a concretized sug-DAG where a package had been patched by a dependent, and the dependent was not in the DAG, we would not read in all patches correctly. - Rather than looking up patches in the DAG, we look them up globally from an index created from the entire repository. - The patch cache is a bit tricky for several reasons: - we have to cache information from packages, specifically, the patch level and working directory. - FilePatches need to know which package owns them, so that they can figure out where the patch lives. The repo can change locations from run to run, so we have to store relative paths and restore them when the cache is reloaded. - Patch files can change underneath the cache, because repo indexes only update on package changes. We currently punt on this -- there are stub methods for needs_update() that will need to check patch files when packages are loaded. There isn't an easy way to do this at global indexing time without making the FastPackageChecker a lot slower. This is TBD for a future commit. - Currently, the same patch can only be used one way in a package. That is, if it appears twice with different level/working_dir settings, bad things will happen. There's no package that current uses the same patch two different ways, so we've punted on this as well, but we may need to fix this in the future by moving a lot of the metdata (level, working dir) to the spec, and *only* caching sha256sums in the PatchCache. That would require some much more complicated tweaks to the Spec, so we're holding off on that til later. - This required patches to be refactored somewhat -- the difference between a UrlPatch and a FilePatch is still not particularly clean.
2018-12-30ProviderIndex uses json instead of YAMLTodd Gamblin4-21/+18
- indexes should use json, not YAML, to optimize for speed - only use YAML in human-editable files - this makes ProviderIndex consistent with other indexes
2018-12-30repo: refactor indexes to be more extensibleTodd Gamblin1-95/+132
- virtual provider cache and tags were previously generated by nearly identical but separate methods. - factor out an Indexer interface for updating repository caches, and provide implementations for each type of index (TagIndex, ProviderIndex) so that more can be added if needed. - Among other things, this allows all indexes to be updated at once. This is an advantage because loading package files is the real overhead, and building the indexes once the packages are loaded is trivial. We avoid extra bulk read-ins by generating all package indexes at once. - This can be extended for dependents (reverse dependencies) and patches later.
2018-12-30patches: clean up patch.py, directives, and package class propertiesTodd Gamblin6-99/+119
- cleanup patch.py: - make patch.py constructors more understandable - loosen coupling of patch.py with package - in Package: make package_dir, module, and namespace class properties - These were previously instance properties and couldn't be called from directives, e.g. in patch.create() - make them class properties so that they can be used in class definition - also add some instance properties to delegate to class properties so that prior usage on Package objects still works
2018-12-25tests: run 'test_compiler_add' serially in Python 2 (#10127)Massimiliano Culpo1-1/+9
* Don't spawn processes during python 2.6/2.7 'test_compiler_add' unit tests * Travis seems to choke on the excessive parallelism in `compiler find`
2018-12-25environment: ignore invalid files names in var/spack/environments (#10198)Denis Davydov3-5/+20
2018-12-23bugfix: handle unicode properly in spack.util.executable (#10186)Todd Gamblin2-18/+37
- When returning string output, use text_type and decode utf-8 in Python 2 instead of using `str` - This properly handles unicode, whereas before we would pass bad strings to colify in `spack blame` when reading git output - add a test that round-trips some unicode through an Executable object
2018-12-21Remove /nfs/tmp2 from default build_stage locations (#10170)Elsa Gonsiorowski, PhD1-6/+4
* Remove /nfs/tmp2 from default configuration * /nfs/tmp2 is going away from LC... and doesn’t exist for the rest of the world. * update documentation to remove /nfs/tmp2 as well
2018-12-20Performance improvements for CDash reporterZack Galbreath1-9/+12
* Record build output as an array of lines rather than concatenating to a single large string. * Use string.find to avoid running re.search on every line of output.
2018-12-20Include accurate build time in CDash reportZack Galbreath1-6/+10
2018-12-20Report current git commit of Spack to CDashZack Galbreath1-6/+23
When using the CDash reporter, upload a Update.xml file that indicates the hash of Spack's current git commit.
2018-12-20Make sure to urlencode cdash submit parametersScott Wittenburg1-3/+9
2018-12-20Get buildId from CDash at submit timeZack Galbreath1-2/+21
Pass extra data when submitting to CDash. If CDash responds with a buildId, construct and display helpful URL that links to the uploaded report.
2018-12-20Allow more customization for CDash reporterZack Galbreath6-22/+61
Add new command line arguments to `spack install` that allow users to set the build name, site name, and track in their CDash report.
2018-12-20Avoid UnboundLocalErrorZack Galbreath1-0/+1
Make sure cdash_phase is defined before referring to it.
2018-12-19env: all commands that disambiguate specs should be env-awareTodd Gamblin19-26/+55
- some commands were missed in the rollout of spack environments - this makes all commands that need to disambiguate specs restrict the disambiguation to installed packages in the active environment, as users would expect