summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2020-04-22spack info: replace "True, False" with "on, off" (#16235)Massimiliano Culpo1-4/+2
fixes #16184
2020-04-22update compiler config with bootstrapped compiler when already installed ↵Greg Becker5-4/+38
(#16221) Update compiler config with bootstrapped compiler when it was already installed and added config defaults to code so mutable_config test fixture works.
2020-04-22spack environments documentation: -E -> -e (#16242)George Hartzell1-2/+2
To specify an environment for a comment, the user can specify "spack -e <env>". The documentation incorrectly specified "-E" (which is actually used to ignore any implicit use of environments).
2020-04-21Doc: Conflict Msg Pkg Guide (#16201)Axel Huebl1-3/+7
Document the nice `msg=` argument in conflicts. Super useful for less generic conflict guidance.
2020-04-21compiler wrappers: don't override -isystem with -I (#16077)Andrew W Elble2-14/+102
If the Spack compiler wrapper encounters any "-isystem" option, then when adding include directories for Spack dependencies, Spack will use "-isystem" instead of "-I". This prevents Spack-generated "-I" options from overriding the "-isystem" options generated by the build system. To ensure that build-system "-isystem" directories are searched first, Spack places all of its inserted "-isystem" directories after. The new ordering of -isystem includes is: * -isystem from build system (not system directories) * -isystem from Spack * -isystem from build system (for directories like /usr/include) The prior order of "-I" arguments is preserved (although as of this commit Spack no longer generates -I if -isystem is detected): * -I from build system (not system directories) * -I from Spack (only if there are no "-isystem" options) * -I from build system (for directories like /usr/include)
2020-04-20autotools: remove FORCE_UNSAFE_CONFIGURE from docs, commands, code (#16145)Todd Gamblin2-11/+4
Since #16132, we've consolidated the setting of FORCE_UNSAFE_CONFIGURE to `autotools.py`, so we don't need to use it in packages like `coreutils`, in our commands, or in our container recipes. - [x] Remove FORCE_UNSAFE_CONFIGURE from packages - [x] Remove FORCE_UNSAFE_CONFIGURE from container recipes - [x] Remove FORCE_UNSAFE_CONFIGURE from `spack ci` command
2020-04-19Typo: vew -> view (#16155)George Hartzell1-1/+1
2020-04-18autotools: set FORCE_UNSAFE_CONFIGURE to 1 in autotools.py (#16132)Diego Magdaleno1-0/+15
This commit sets the `FORCE_UNSAFE_CONFIGURE` environment variable to 1 in autotools builds. We see a lot of builds popping up and complaining about `FORCE_UNSAFE_CONFIGURE`. This behavior is not actually part of `autoconf` per se. It comes from this patch to `mknod.m4`, which is used by a lot of autoconf builds: * https://lists.gnu.org/archive/html/bug-gnulib/2010-07/msg00282.html Which originated from this problem that someone had on AIX: * https://lists.gnu.org/archive/html/bug-gnulib/2010-07/msg00279.html The gist of the problem seems to be that they want to check whether `mknod` can do something as root, but instead of checking whether they're running as root and using `su` or something to test this, they just made it harder to run `configure` as root. This seems very ad hoc and this is one of many checks that are run as root in `configure`. Many of them run before this check, so it's not clear that the `FORCE_UNSAFE_CONFIGURE` thing is even preventing bad things from happening. So: 1. This only happens in `autotools` builds, so we should go ahead and put it into `autotools.py` instead of in the global build environment, and 2. The variable does too little and provides a false sense of security in the first place, so we'll just disable it and avoid the nuisance. If we really feel strongly about this we can put some warnings in Spack about running as root, but at the top level, not in the middle of an already running script like `configure`.
2020-04-17remove files accidentally committed with 0.14.0 (#16138)Todd Gamblin3-11712/+0
2020-04-17AutotoolsPackage: add aclocal for all build deps (#15784)Adam J. Stewart1-5/+5
2020-04-17Language-specific PIC flags (#15474)Adam J. Stewart12-77/+300
* Language-specific PIC flags * Add tests for every compiler flag * Fix bad rebase * pic_flag -> cxx_pic_flag
2020-04-16SourceForge: Mirror Mixin (#16112)Axel Huebl3-0/+64
* SourceForge: Mirror Mixin Add a mixing class for direct `CNAME`s to sourceforge mirrors. Since the main gateway servers are often down, this could reduce timeouts and fetch errors for sourceforge.net hosted software. * SourceForge: unspectacular mirror replacement add mirrors to all sourceforge packages with trivial download logic. tested fetch of latest version of each of these packages with various mirrors before committing. * SourceForge: xz the author homepage is chronocially overrun and this is the offical upload with many mirrors.
2020-04-16macos: use DYLD_FALLBACK_LIBRARY_PATH instead of DYLD_LIBRARY_PATHTodd Gamblin3-2/+6
`DYLD_LIBRARY_PATH` can frequently break builtin macOS software when pointed at Spack libraries. This is because it takes *higher* precedence than the default library search paths, which are used by system software. `DYLD_FALLBACK_LIBRARY_PATH`, on the other hand, takes lower precedence. At first glance, this might seem bad, because the software installed by Spack in an environment needs to find *its* libraries, and it should not use the defaults. However, Spack's isntallations are always `RPATH`'d, so they do not have this problem. `DYLD_FALLBACK_LIBRARY_PATH` is thus useful for things built in an environment that need to use Spack's libraries, that don't set *their* RPATHs correctly for whatever reason. We now prefer it to `DYLD_LIBRARY_PATH` in modules and in environments because it helps a little bit, and it is much less intrusive.
2020-04-16Bugfix: For spack buildcache check, Handle specs where only the hash is (#15663)Marc Allen1-1/+1
provided (#15662). Prior to this fix, the checked Spec object would not be populated, and concretization would fail. Co-authored-by: Marc Allen <mrcall@amazon.com>
2020-04-15Merge branch 'releases/v0.14' into developTodd Gamblin1-1/+1
2020-04-15Make tags case insensitiveMichael Kuhn2-0/+10
Currently, tags are case sensitive, which is unintuitive: ```console $ spack list -t hpc ==> 2 packages. nek5000 nektools $ spack list -t HPC ==> 1 packages. mfem $ spack list -t Hpc ==> 0 packages. ``` This change makes them case insensitive: ```console $ spack list -t hpc ==> 3 packages. mfem nek5000 nektools $ spack list -t HPC ==> 3 packages. mfem nek5000 nektools $ spack list -t Hpc ==> 3 packages. mfem nek5000 nektools ```
2020-04-15version bump: 0.14.2Todd Gamblin1-1/+1
2020-04-15bugfix: spack test should not output [+] for mock installs (#15609)Todd Gamblin1-3/+2
`spack test` has a spurious '[+] ' in the output: ``` lib/spack/spack/test/install.py .........[+] ...... ``` Output is properly suppressed: ``` lib/spack/spack/test/install.py ............... ```
2020-04-15`spack install` terminal output handling in foreground/background (#15723)Todd Gamblin7-224/+1155
Makes the following changes: * (Fixes #15620) tty configuration was failing when stdout was redirected. The implementation now creates a pseudo terminal for stdin and checks stdout properly, so redirections of stdin/out/err should be handled now. * Handles terminal configuration when the Spack process moves between the foreground and background (possibly multiple times) during a build. * Spack adjusts terminal settings to allow users to to enable/disable build process output to the terminal using a "v" toggle, abnormal exit cases (like CTRL-C) could leave the terminal in an unusable state. This is addressed here with a special-case handler which restores terminal settings. Significantly extend testing of process output logger: * New PseudoShell object for setting up a master and child process and configuring file descriptor inheritance between the two * Tests for "v" verbosity toggle making use of the added PseudoShell object * Added `uniq` function which takes a list of elements and replaces any consecutive sequence of duplicate elements with a single instance (e.g. "112211" -> "121") Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-04-15database: maintain in-memory consistency on remove (#15777)Massimiliano Culpo2-1/+21
The performance improvements done in #14693 where leaving the DB in an inconsistent state when specs were removed from it. This PR updates the DB internal state whenever the DB is written to a file. Note that we still cannot properly enumerate installed dependents, so there is a TODO in this code. Fixing that will require the dependents dictionaries in specs to be re-keyed (either by hash, or not keyed at all -- a list would do). See #11983 for details.
2020-04-15performance: add a verification file to the database (#14693)Andrew W Elble3-3/+52
Reading the database repeatedly can be quite slow. We need a way to speed up Spack when it reads the DB multiple times, but the DB has not been modified between reads (which is nearly all the time). - [x] Add a file containing a unique uuid that is regenerated at database write time. Use this uuid to suppress re-parsing the database contents if we know a previous uuid and the uuid has not changed. - [x] Fix mutable_database fixture so that it resets the last seen verifier when it resets. - [x] Enable not rereading the database immediately after a write. Make the tests reset the last seen verifier in between tests that use the database fixture. - [x] make presence of uuid module optional
2020-04-15Remove DB conversion of old index.yaml (#15298)Peter Scheibel1-84/+45
Removed the code that was converting the old index.yaml format into index.json. Since the change happened in #2189 it should be considered safe to drop this (untested) code.
2020-04-15Environments/views: only override spec prefix for non-external packages (#15475)Peter Scheibel2-1/+49
* only override spec prefix for non-external packages * add test that environment shell modifications respect explicitly-specified prefixes for external packages * add clarifying comment
2020-04-15Source devnull in environment_after_sourcing_files (closes #15775) (#15791)Jonathon Anderson1-1/+2
spack.util.environment_after_sourcing_files compares the local environment against a shell environment after having sourced a file; but this ends up including the default shell profile and rc, which might differ from the local environment. To change this, compare against the default shell environment, expressed here as 'source /dev/null'.
2020-04-15bugfix: TERM may not be in the environment on Cray (#15630)Todd Gamblin1-3/+7
2020-04-15Blacklist Lmod variable modifications when sourcing files (#15778)Massimiliano Culpo3-1/+24
fixes #15775 Add all the variables listed here: https://lmod.readthedocs.io/en/latest/090_configuring_lmod.html to the list of those blacklisted when constructing environment modifications by sourcing files.
2020-04-15Tests: Mirror Mixin ClassesAxel Huebl1-0/+69
2020-04-15x.org: mirror mixinAxel Huebl2-0/+38
According to my nightly CI/CD tests, x.org is another large provider of software in common build chains that is often down. Added a hand-selected amount of mirrors that is well up-to-sync. Tested with `util-macros` that has a quite "recent" patch release. Other packages to follow in an individual PR.
2020-04-15`spack install` terminal output handling in foreground/background (#15723)Todd Gamblin7-224/+1155
Makes the following changes: * (Fixes #15620) tty configuration was failing when stdout was redirected. The implementation now creates a pseudo terminal for stdin and checks stdout properly, so redirections of stdin/out/err should be handled now. * Handles terminal configuration when the Spack process moves between the foreground and background (possibly multiple times) during a build. * Spack adjusts terminal settings to allow users to to enable/disable build process output to the terminal using a "v" toggle, abnormal exit cases (like CTRL-C) could leave the terminal in an unusable state. This is addressed here with a special-case handler which restores terminal settings. Significantly extend testing of process output logger: * New PseudoShell object for setting up a master and child process and configuring file descriptor inheritance between the two * Tests for "v" verbosity toggle making use of the added PseudoShell object * Added `uniq` function which takes a list of elements and replaces any consecutive sequence of duplicate elements with a single instance (e.g. "112211" -> "121") Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-04-14Package extensions: fixed a link in docs (#16040)Massimiliano Culpo3-7/+9
* Moved link to the right place in the docs * Fixed a few minor issues in extensions docs Fixed a typo, added a subsubsection for better navigation, reworded "modules in Python" as "Python packages"
2020-04-14sourceware.org: mirror urls (#15992)Axel Huebl2-0/+38
sourceware.org is often quite overrun and times out or results in certificate errors. Since libffi, bzip2, elfutils, etc. are quite fundamental in build chains, lets add some official mirrors. libffi, bzip2, elfutils, lvm2, valgrind: add mirrors
2020-04-12database: maintain in-memory consistency on remove (#15777)Massimiliano Culpo2-1/+21
The performance improvements done in #14693 where leaving the DB in an inconsistent state when specs were removed from it. This PR updates the DB internal state whenever the DB is written to a file. Note that we still cannot properly enumerate installed dependents, so there is a TODO in this code. Fixing that will require the dependents dictionaries in specs to be re-keyed (either by hash, or not keyed at all -- a list would do). See #11983 for details.
2020-04-10fujitsu compiler: Delete custom environment function. (#15988)Tomoki, Karatsu1-4/+0
2020-04-09NAG compiler fix: skip implicit RPATH detection (#15902)Peter Scheibel1-2/+6
* Skip collection of compiler link paths if compiler does not define a verbose flag * modules config bug: allow user to configure a compiler without an explicit entry for loaded modules
2020-04-07pipelines: Document dynamic generation limitationsScott Wittenburg1-1/+23
2020-04-07Add build number to Arm compiler version identification (#15809)LPettey-Arm2-2/+22
* Add capability for detecting build number for Arm compilers * Fixing fleck8 errors and updating test_arm_version_detection function for more detailed Arm compielr version detection * Ran flake8 locally and corrected errors * Altering Arm compielr version check to remove else clause and be more consistent with other compielr version checks. Added test case so both the 'if' and 'else' conditionals of the Arm compiler version check have a test case Co-authored-by: EC2 Default User <ec2-user@ip-172-31-7-135.us-east-2.compute.internal>
2020-04-06Docs: fix Spack Environments command (#15891)Adam J. Stewart1-1/+1
2020-04-06Blacklist Lmod variable modifications when sourcing files (#15778)Massimiliano Culpo3-1/+24
fixes #15775 Add all the variables listed here: https://lmod.readthedocs.io/en/latest/090_configuring_lmod.html to the list of those blacklisted when constructing environment modifications by sourcing files.
2020-04-06Source devnull in environment_after_sourcing_files (closes #15775) (#15791)Jonathon Anderson1-1/+2
spack.util.environment_after_sourcing_files compares the local environment against a shell environment after having sourced a file; but this ends up including the default shell profile and rc, which might differ from the local environment. To change this, compare against the default shell environment, expressed here as 'source /dev/null'.
2020-04-05verify `other.exe` attr exists before use in __eq__ (#15882)Stephen McDowell1-1/+1
2020-04-03Add option "--first" for "spack load" (#15622)G-Ragghianti4-7/+34
* Implemented --first option for "spack load" * added test for "spack load --first" Co-authored-by: gragghia <gragghia@localhost.localdomain>
2020-04-03Environments/views: only override spec prefix for non-external packages (#15475)Peter Scheibel2-1/+49
* only override spec prefix for non-external packages * add test that environment shell modifications respect explicitly-specified prefixes for external packages * add clarifying comment
2020-04-02Add commands to facilitate Spack/Python/OS reporting (#15834)Adam J. Stewart4-2/+46
* Add --version arg to spack python command * Add `spack debug report` command
2020-04-01Spelling fixes (#15805)Greg Sjaardema7-13/+13
* Spelling fixes * spelling fixes * spelling fixes * spelling fixes * spelling fix * spelling fix * spelling fix * spelling fixes * spelling fix
2020-03-31packages.yaml: allow virtuals to specify buildable: false (#14934)Greg Becker9-153/+155
Currently, to force Spack to use an external MPI, you have to specify `buildable: False` for every MPI provider in Spack in your packages.yaml file. This is both tedious and fragile, as new MPI providers can be added and break your workflow when you do a git pull. This PR allows you to specify an entire virtual dependency as non-buildable, and specify particular implementations to be built: ``` packages: all: providers: mpi: [mpich] mpi: buildable: false paths: mpich@3.2 %gcc@7.3.0: /usr/packages/mpich-3.2-gcc-7.3.0 ``` will force all Spack builds to use the specified `mpich` install.
2020-03-31Update link for codecov's browser extensions (#15779)Massimiliano Culpo1-2/+2
2020-03-30CUDA 10.1+ supports ICC 19.0 (#15764)Axel Huebl1-2/+2
Fix the CUDA-ICC conflict check. Simplify the XL 17 check.
2020-03-28Buildcache keys command speed up (#15727)Patrick Gartung1-1/+1
* Limit the spidering to depth=0 for keys * depth=0 is default argument
2020-03-25provider index: removed import from + refactored a few parts (#15570)Massimiliano Culpo2-130/+179
Removed provider_index use of 'import from' and refactored a few routines to a further subclassing of _IndexBase for implementing user defined bindings of provider specs.
2020-03-24spack.repo: remove "import from" statements (#15505)Massimiliano Culpo3-39/+71
spack.repo: remove "import from" statements