summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2019-03-19Fix help text for --no-checksum. (#10932)Chris Green1-1/+1
2019-03-14Ensure that every package has a description (#10896)Adam J. Stewart1-0/+8
2019-03-14Support VISUAL environment variable for editing. (#10898)Chris Green1-20/+26
If the user has set the environment variable VISUAL, it will be used in preference to EDITOR for all Spack editing activities. If VISUAL is not set or fails (perhaps due to a lack of graphical editing capabilities),EDITOR will be used instead. We fall back to one of several common editors if neither bears fruit. This feature has been tailored to: * Provide identical behavior to the previous implementation in the case that VISUAL is not set. * Not require any change to code utilizing the editor feature. * Follow usual UNIX behavior concerning VISUAL and EDITOR.
2019-03-12Do not use `string` module to be compatible with python 3 (#248) (#10667)Tristan Carel1-3/+2
`string.find` is not part of Python 3 anymore.
2019-03-12find_versions_of_archive: expand link search (#10758)Justin Stanley1-1/+2
2019-03-12Fix clearing EnvironmentModifications with python2 (#10791)Nick Forrington2-1/+8
* Fix clearing EnvironmentModifications with python2 * Add EnvironmentModifications::clear unit test Use re-assignment rather than del to clear array * Fix flake issues
2019-03-11format change and typo in doc (#10848)Shahzeb Siddiqui1-2/+2
2019-03-11Improved detection of Clang versions (#10316)Massimiliano Culpo2-12/+47
Fixes #10191 * Add more regular expressions to detect clang versions that were not being picked up * Add a test for parsing versions from the output of Clang (this does not run Clang, but rather uses example outputs from Clang) * Separate Clang version parsing into its own method (to make it easier to test)
2019-03-08Fix find_headers to also look for C++ headers and Fortran modules (#10798)Michael Kuhn1-3/+6
Currently, only C headers are considered, causing build failures for packages depending on, e.g., netcdf-fortran and xerces-c. Additionally, the regex used to look for the include path component did not consider word boundaries, causing false matches.
2019-03-07Create option to build compilers as needed (#10761)Greg Becker6-65/+131
* Create option to build missing compilers and add them to config before installing packages that use them * Clean up kwarg passing for do_install, put compiler bootstrapping in separate method
2019-03-06Make is a required dependency of Spack (#10386)Adam J. Stewart1-4/+5
Update documentation on getting started to mention that a "make" executable is required to build packages with Spack.
2019-03-01This fixes a problem where the placeholder path was not in the first rpath ↵Patrick Gartung3-105/+76
entry. * Rework of buildcache creation and install prefix checking using the functions introduced in https://github.com/spack/spack/pull/9199 Instead of replacing rpaths with placeholder and then checking strings, make use of the functions relocate.is_recocatable and relocate.is_file_relocatable to decide if a package needs the allow-root option. This fixes a problem where the placeholder path was not in the first rpath entry. This was seen in c++ libraries and binaries because the compiler was outside the spack install base path and always appears first in the rpath. Instead of checking the first rpath entry, all rpaths have the placeholder path and the old install path (if it exists) replaced with the new install path. * flake8
2019-02-28Added a sub-command to show if packages are relocatable (#9199)Massimiliano Culpo10-25/+309
* Added the `spack buildcache preview` sub-command This is similar to `spack spec -I` but highlights which nodes in a DAG are relocatable and which are not. spec.tree has been generalized a little to accept a status function, instead of always showing the install status The current implementation works only for ELF, and needs to be generalized to other platforms. * Added a test to check if an executable is relocatable or not This test requires a few commands to be present in the environment. Currently it will run only under python 3.7 (which uses Xenial instead of Trusty). * Added tests for the 'buildcache preview' command. * Fixed codebase after rebase * Fixed the list of apt addons for Python 3.7 in travis.yaml * Only check ELF executables and shared libraries. Skip checking virtual or external packages. (#229) * Fixed flake8 issues * Add handling for macOS mach binaries (#231)
2019-02-28Fix shell integration with environment-modules@4 (#10736)Michael Kuhn1-1/+1
2019-02-27Fix 'make test' detection when LANG is not in English (#10499)Adam J. Stewart1-1/+5
2019-02-27Update environment-modules package (#10717)Massimiliano Culpo1-1/+1
The environment modules package has been updated to include versions up to 4.0.0. The url of the package and the homepage have been updated accordingly. The `spack bootstrap` command now builds version 3.2.10 of the environment-modules package, and will do until #10708 is fixed.
2019-02-26Use Package.headers for -I options (#10623)Massimiliano Culpo6-25/+167
This restores the use of Package.headers when computing -I options for building a package that was added in #8136 and reverted in #10604. #8136 used utility logic that located all header files in an installation prefix, and calculated the -I options as the immediate roots containing those header files. In some cases, for a package containing a directory structure like prefix/ include/ ex1.h subdir/ ex2.h dependents may expect to include ex2.h relative to 'include', and adding 'prefix/include/subdir' as a -I was causing errors, in particular if ex2.h has the same name as a system header. This updates header utility logic to by default return the base "include" directory when it exists, rather than subdirectories. It also makes it possible for package implementers to override Package.headers to return the subdirectory when it is required (for example with libxml2).
2019-02-26Add libhio v1.4.1.3, 1.4.1.1 (#10699)Daniel Topa1-1/+1
- Switch all libhio tarball listings to sha256 checksums - Correct typo in alert message in /lib/spack/spack/util/module_cmd.py: fucntion -> function https://github.com/hpc/libhio/releases Released 2019-02-01 Verification builds on LANL Darwin: **Intel Xeon** Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 40 On-line CPU(s) list: 0-39 Thread(s) per core: 2 Core(s) per socket: 10 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 63 Model name: Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz Stepping: 2 CPU MHz: 1198.779 CPU max MHz: 3300.0000 CPU min MHz: 1200.0000 BogoMIPS: 5193.70 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 25600K NUMA node0 CPU(s): 0-9,20-29 NUMA node1 CPU(s): 10-19,30-39 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm cpuid_fault epb invpcid_single pti intel_ppin tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts ` [+] /scratch/users/dantopa/new-spack/strawman.pr.libhio/opt/spack/linux-centos7-x86_64/gcc-4.8.5/libhio-1.4.1.3-s4fnmesfp65trhks5qi3it5p73ssfpsp [+] /scratch/users/dantopa/new-spack/strawman.pr.libhio/opt/spack/linux-centos7-x86_64/gcc-4.8.5/libhio-1.4.1.2-fkgh5vqpijvwqywffmokgmsglqxwfrtl ` **Arm** Architecture: aarch64 Byte Order: Little Endian CPU(s): 256 On-line CPU(s) list: 0-255 Thread(s) per core: 4 Core(s) per socket: 32 Socket(s): 2 NUMA node(s): 2 Model: 0 BogoMIPS: 400.00 L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 32768K NUMA node0 CPU(s): 0-127 NUMA node1 CPU(s): 128-255 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics cpuid asimdrdm ` [+] /scratch/users/dantopa/new-spack/strawman.pr.libhio/opt/spack/linux-rhel7-aarch64/gcc-4.8.5/libhio-1.4.1.3-q6nnwiy6bi7ktnghdsngwamom23zpmgy [+] /scratch/users/dantopa/new-spack/strawman.pr.libhio/opt/spack/linux-rhel7-aarch64/gcc-4.8.5/libhio-1.4.1.2-y6nwovff3qbdy242zc4x2toloz6xpcvm ` 2019-02-25 Signed-off-by: Daniel Topa <dantopa@lanl.gov>
2019-02-25Add /hash to spack help --spec (#10691)Adam J. Stewart1-1/+5
* Add /hash to spack help --spec * Make /hash gray, add ^/hash
2019-02-21CPATH warning: downgrade to debug message (#10675)Peter Scheibel1-3/+3
Spack warns users when a dependency package updates CPATH. This warning message is generating bug reports and alarm in cases where there is no problem. For now this downgrades the warning message to the debug level, so it only shows up if something goes wrong for the user and they ask for more information from Spack.
2019-02-21release workflow: Add build scripts for jobs and means to upload pkgsScott Wittenburg1-0/+212
2019-02-21release workflow: Add spack command for generating the .gitlab-ci.ymlScott Wittenburg4-0/+823
This spack command adds a new schema for a file which describes the builder containers available, along with the compilers availabe on each builder. The release-jobs command then generates the .gitlab-ci.yml file by first expanding the release spec set, concretizing each spec (in an appropriate docker container if --this-machine-only argument is not provided on command line), and then combining and staging all the concrete specs as jobs to be run by gitlab.
2019-02-21buildcache: Add sub-commands needed by release workflowScott Wittenburg5-79/+599
Adds four new sub-commands to the buildcache command: 1. save-yaml: Takes a root spec and a list of dependent spec names, along with a directory in which to save yaml files, and writes out the full spec.yaml for each of the dependent specs. This only needs to concretize the root spec once, then indexes it with the names of the dependent specs. 2. check: Checks a spec (via either an abstract spec or via a full spec.yaml) against remote mirror to see if it needs to be rebuilt. Comparies full_hash stored on remote mirror with full_hash computed locally to determine whether spec needs to be rebuilt. Can also generate list of specs to check against remote mirror by expanding the set of release specs expressed in etc/spack/defaults/release.yaml. 3. get-buildcache-name: Makes it possible to attempt to read directly the spec.yaml file on a remote or local mirror by providing the path where the file should live based on concretizing the spec. 4. download: Downloads all buildcache files associated with a spec on a remote mirror, including any .spack, .spec, and .cdashid files that might exist. Puts the files into the local path provided on the command line, and organizes them in the same hierarchy found on the remote mirror This commit also refactors lib/spack/spack/util/web.py to expose functionality allowing other modules to read data from a url.
2019-02-21add CombinatorialSpecSet class for taking cross-products of Specs.Todd Gamblin5-0/+623
- add CombinatorialSpecSet in spack.util.spec_set module. - class is iterable and encaspulated YAML parsing and validation. - Adjust YAML format to be more generic - YAML spec-set format now has a `matrix` section, which can contain multiple lists of specs, generated different ways. Including: - specs: a raw list of specs. - packages: a list of package names and versions - compilers: a list of compiler names and versions - All of the elements of `matrix` are dimensions for the build matrix; we take the cartesian product of these lists of specs to generate a build matrix. This means we can add things like [^mpich, ^openmpi] to get builds with different MPI versions. It also means we can multiply the build matrix out with lots of different parameters. - Add a schema format for spec-sets
2019-02-15Dependency libs: filter system paths and always add lib dir (#10622)Peter Scheibel1-7/+16
Fixes #10617 Fixes #10624 Closes: #10619 #8136 dependended entirely on spec.libs to retrieve library directories from dependencies. By default this function only retrieves libraries if their name is something like lib<package> (e.g. "libfoo.so" for a package called "Foo"). This unconditionally adds lib/lib64 directories for each dependency as link/rpath directories. This also filters system paths from link/rpaths/include directories and removes duplicated paths that #8136 could add.
2019-02-15Fix bug getting specs from build caches (#9600)Javier Cervantes1-2/+2
2019-02-14CDash: allow installing from spec.yaml (#10565)Scott Wittenburg2-1/+48
If the -f <specyamlfile> argument to install is used (rather than providing package specs on the command line), CDash throws an exception due to missing the installation command (the packages targeted for install). This fixes that behavior so CDash reporting succeeds in either case.
2019-02-14Removed pkg.headers.directories from the include list (#10604)Massimiliano Culpo3-15/+13
fixes #10601 Due to a bug this attribute is wrong for packages that use directories as namespaces. For instance it will add "<boost-prefix>/include/boost" instead of "<boost-prefix>/include" to the include path. As a minor addition a few loops in the compiler wrappers have been simplified.
2019-02-13Dynamic library/include paths (#8136)Peter Scheibel8-248/+261
Fixes #7855 Closes #8070 Closes #2645 When searching for library directories (e.g. to add "-L" arguments to the compiler wrapper) Spack was only trying the "lib/" and "lib64/" directories for each dependency install prefix; this missed cases where packages would install libraries to subdirectories and also was not customizable. This PR makes use of the ".headers" and ".libs" properties for more-advanced location of header/library directories. Since packages can override the default behavior of ".headers" and ".libs", it also allows package writers to customize. The following environment variables which used to be set by Spack for a package build have been removed: * Remove SPACK_PREFIX and SPACK_DEPENDENCIES environment variables as they are no-longer used * Remove SPACK_INSTALL environment variable: it was not used before this PR
2019-02-13Bug Fix in permission setter (#10584)abernede1-2/+3
* fix permission setter Fix a typo in islink test when applied to files. * os.walk explicitly set not to follow links The algorithm strongly rely on not following links.
2019-02-13Allow tty output to be timestamped (#10554)sknigh5-6/+103
* Spack debug output now includes microsecond-granularity timestamps. * Timestamps can also be enabled with the `--timestamp` command line argument.
2019-02-13enh: allow time like HH:MM in date strings. (#10034)Matthias Wolf2-1/+8
2019-02-12Enhancement to module autoload documentation (#10310)George Hartzell1-1/+15
* Note that `none` is the default for lmod autoload Save a bit of confusion by *explicitly* pointing out that `none` is the default value for autoload in the lmod module file generator. * Add a tip re building software externally Add a tip about using `autoload: all` when building packages outside of the tree that use artifacts (e.g. libraries, includes) within the tree.
2019-02-12Allow secondary generators when building with CMake. (#9324)Chris Green2-4/+50
CMake supports the notion of secondary generators which provide extra information to (e.g.) IDEs over and above that normally provided by the primary generator. Spack only supports the 'Unix Makefiles' and 'Ninja' primary generators but was not parsing out the primary generator when a secondary generator was also included (e.g. for a generator attribute like 'Codeblocks - Ninja'). This adds a regex for extracting the primary generator for validation. Since the secondary generator is irrelevant to a Spack build, it is passed on to CMake without further validation.
2019-02-08CudaPackage: fix wrong version range (#10551)Denis Davydov1-1/+1
2019-02-07log-parse: fix error message when no error lines are found (#10543)Massimiliano Culpo1-1/+1
2019-02-06Add x86_64 as target to cray platform (#10369)Mario Melara1-0/+4
Add x86_64 as a target for the Cray platform, and also designate it as the default front_end target.
2019-02-06CUDA compiler conflicts for Linux (#10460)Nichols A. Romero1-14/+69
* CUDA compiler conflicts for Linux. * Add Volta and Turing GPUs. * Add mandatory conflict for Volta and Turing GPUs. * Revert "CUDA compiler conflicts for Linux." This reverts commit 7d4ff654ac53aad272c59e9f7f8bb3fbb32bcec4. * Compiler conflicts introduced from previous commit into CUDA packaged moved and integrated into CUDA build system. * More conversative with compiler conflicts for cuda 10.0.130, since I don't know what will happen with future cuda 10.x releases. * Correct off-by-one errors in clang conflicts for x86_64 Linux. * No restrictions on Apple Clang compiler until we are able to distinguish Xcode clang from github clang more easily. Note to fix this in the future. * Change comment to clarify that github clang refers to LLVM clang. * Fix and simplify index range. * Fix overlapping conflicts for CUDA 10.0.130 * Removed extra ^cuda from conflict.
2019-01-29debug output includes modulecmd output (#9476)Peter Scheibel1-7/+28
Debug output now includes the output of modulecmd executions. Only output module content when a failure occurs; always report when a module is loaded/unloaded.
2019-01-24Environments: fix bug for install of external packages (#10437)Peter Scheibel1-0/+1
"spack install" will install all packages added to the current environment. When this included external packages, the environment update would fail because it would attempt to copy log files that were only generated if Spack handled the install itself. This skips that step for external packages.
2019-01-24Allow `spack install --overwrite` for nonexistent or multiple packages (#9201)Greg Becker2-25/+78
* Allow overwrite nonexistent and multiple packages initial implementation give one prompt to users instead of a prompt per spec testing * flake * bugfix: install overwrite check each spec against installed * python3 compliance for filter/map
2019-01-14tutorial: update tutorial materials for ECP19Todd Gamblin1-14/+12
2019-01-13version bump: v0.12.1Gregory Becker1-1/+1
2019-01-12cc: clean up cray compilers, fix issues with case-insensitive filesystems ↵Adam J. Stewart9-9/+6
(#10323) * Remove Cray CC compilers causing problems on case-insensitive filesystems * cray -> cce * Ensure that compiler-specific directory comes first in build-env * Point to compiler-specific symlinks
2019-01-11Binary caching bugfix: symlink relocation (#10073)Greg Becker3-9/+131
Binary caches of packages with absolute symlinks had broken symlinks. As a stopgap measure, #9747 addressed this by replacing symlinks with copies of files when creating binary cached packages. This reverts #9747 and instead, either relative-izes the symlink or rewrites the target. If the binary cache is created using '--rel' (as in "spack buildcache create --rel...") then absolute symlinks will be replaced with relative symlinks (in addition to making RPATHs relative as before); otherwise they are rewritten (when the binary cache is unpacked and installed).
2019-01-11enh: `buildcache list` should behave similar to `find` (#10052)Matthias Wolf1-15/+10
The current output of buildcache list is very verbose and I feel like some details are getting lost. By making the output similar to find, I think users will be able to get a better overview of what is stored in the cache.
2019-01-11dealii: fix concretization of xsdk package (#10288)Denis Davydov1-0/+11
* dealii: fix concretization of xsdk package * tests: add concretization tests for deal.II and xSDK, which are often broken due to limitations in the concretizer * use pytest.mark.parametrize
2019-01-10spack versions: only list safe versions (#10004)Adam J. Stewart2-10/+27
* spack versions: only list safe versions * Add unit tests for spack versions -s
2019-01-09Allow combinatorial projections in views (#9679)Greg Becker14-147/+500
Allow customizing views with Spec-formatted directory structure Allow views to specify projections that are more complicated than merging every package into a single shared prefix. This will allow sites to configure a view for the way they want to present packages to their users; for example this can be used to create a prefix for each package but omit the DAG hash from the path. This includes a new YAML format file for specifying the simplified prefix for a spec in a view. This configuration allows the use of different prefix formats for different specs (i.e. specs depending on MPI can include the MPI implementation in the prefix). Documentation on usage of the view projection configuration is included. Depending on the projection configuration, paths are not guaranteed to be unique and it may not be possible to add multiple installs of a package to a view.
2019-01-09tutorial basics section: fix gcc install version (#10298)Greg Becker1-1/+1