summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2018-11-09env: add -e as global spack argument, make `spack -e <env> spec` workTodd Gamblin6-40/+64
- add -E/--exact-env instead of --use-env-repo - simplify env handling in `spack find`
2018-11-09env: add `--env` argument to `spack location`Todd Gamblin2-11/+12
2018-11-09env: add --env argument to `spack find`Todd Gamblin5-9/+52
- add a common argument for `-e/--env` - modify the database to support queries on subsets of hashes - allow `spack find` to be filtered by hashes in an environment
2018-11-09bugfix: identical specs with different DAG hashes don't shadow each otherTodd Gamblin1-3/+5
- logic used in `spack find` was hiding duplicate installations if their hashes were different - short hash doesn't work in this scenario, since specs are structurally identical - ConstraintAction always works on a DB query, so use the DAG hash to ensure uniqueness
2018-11-09env: move main Environment class and logic to `spack.environment`Todd Gamblin5-601/+629
- `spack.environment` is now the home for most of the infrastructure around Spack environments - refactor `cmd/env.py` to use everything from spack.environment - refactor the cmd/env test to use pytest and fixtures
2018-11-09utils: merge spack.environment into spack.util.environmentTodd Gamblin6-634/+628
- `spack.util.environment` is the new home for routines that modify environment variables. - This is to make room for `spack.environment` to contain new routines for dealing with spack environments
2018-11-09env: refactor imports in cmd/env.py and testsTodd Gamblin2-16/+11
2018-11-09env: refactor subparsers in env.pyTodd Gamblin1-121/+116
- Instead of one method with all parsers, each subcommand gets two functions: `setup_<cmd>_parser()` and `environment_<cmd>()` - the `setup_parser()` and `env()` functions now generate the parser based on these and a list of subcommands. - it is now easier to associate the arguments with the subcommand.
2018-11-09env: add spack env command, along with env.yaml schema and testsPeter Josef Scheibel3-0/+987
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09env: preserve command_line as the scope of highest precedencePeter Josef Scheibel2-10/+23
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09env: refactor common argumentsElizabeth Fischer6-58/+108
2018-11-09specs: save/restore concrete & patches when exporting/importing SpecsElizabeth Fischer1-0/+20
2018-11-09specs: _concrete should never be cleared when copying SpecsPeter Josef Scheibel1-2/+2
2018-11-09specs: add convenience function for reading spec dependencies from node dictPeter Josef Scheibel1-0/+9
2018-11-09specs: allow writing full spec (including build deps) to dictPeter Josef Scheibel1-5/+13
2018-11-09env: rename 'spack env' command to 'spack build-env'Peter Josef Scheibel2-6/+6
2018-11-09env: renamed 'environment' testPeter Josef Scheibel1-0/+0
2018-11-06New repo for advanced packaging tutorial (#9711)Peter Scheibel1-35/+11
* modified tutorial packages * update hint in hdf5 tutorial file (typo for suggested argument) * add repo.yaml to tutorial repository * update tutorial docs to refer user to tutorial package repository * flake edits * recommend site scope vs. defaults * you don't specify the repo's name when adding a repo, just the path
2018-11-06Binary caching: remove symlinks, copy files instead (#9747)Peter Scheibel1-3/+5
* omit symlinks and create file copies when making a binary cache of a package * unrelated flake edits involving regexes that recent flake is now angry about
2018-11-06cdash: report clean results to CDash server (#9564)Zack Galbreath4-12/+38
* Record stdout for packages without errors Previously our reporter only stored stdout if something went wrong while installing a package. This prevented us from properly reporting on steps where everything went as expected. * More robustly report all phases to CDash Previously if a phase generated no output it would not be reported to CDash. For example, consider the following output: ==> Executing phase: 'configure' ==> Executing phase: 'build' This would not generate a report for the configure phase. Now it does. * Add test case for CDash reporting clean builds * Fix default directory for CDash reports The default 'cdash_report' directory name was getting overwritten by 'junit-report'. * Upload the build phase first to CDash Older versions of CDash expect Build.xml to be the first file uploaded for any given build. * Define cdash_phase before referring to it
2018-11-06Adapted the code of the non-daemonic pool to recent python versionsMassimiliano Culpo1-7/+19
fixes #9739 The non-daemonic pool relies heavily on implementation details of the multiprocessing package. In this commit we provide an implementation that fits recent python versions.
2018-11-02compilers: add arm compiler detection to SpackNick Forrington1-0/+73
- added arm.py with support for detecting `armclang` and `armflang` Co-authored-by: Srinath Vadlamani <srinath.vadlamani@arm.com>
2018-10-31Parse the ${NAMESPACE} format string in a spec's format method. (#9686)Tim Fuller1-0/+3
This allows installing software on a namespace basis by including ${NAMESPACE} in `install_path_scheme`. e.g., ``` cat ~/.spack/config.yaml config: install_path_scheme: "${ARCHITECTURE}/${NAMESPACE}/${COMPILERNAME}-${COMPILERVER}/${PACKAGE}-${VERSION}-${HASH}" ```
2018-10-31static_to_shared_library: separate options from option values (#9690)Satish Balay1-4/+4
The 'static_to_shared_library' function takes a compiler Executable, which is intended to be invoked with a list of arguments; the arguments must be separated from their values in the list, given the way that 'Executable.__call__' invokes the underlying executable. 'static_to_shared_library' was not doing this, which this commit fixes.
2018-10-30compilers: update clang fortran compiler wrapper selection (#9678)Todd Gamblin1-16/+44
Clang has support for using different fortran compilers with the Clang executable. Spack includes logic to select a compiler wrapper symlink which refers to the fortran executable (since some build systems depend on the name of the compiler, e.g. 'gfortran' or 'flang'). This selection was previously based on the architecture, and chose incorrectly in some situations (e.g. for clang/gfortran on Linux). This replaces architecture-based wrapper selection with a selection that is based on the name of the Fortran compiler executable.
2018-10-30tests: add test for FIXME boilerplate to package_sanity (#9285)Levi Baber1-0/+25
* package_sanity: add test_no_fixme * cleanup & better assert message
2018-10-26config: `spack config blame` now colors filenames in config output (#9656)Todd Gamblin1-1/+11
- it was hard to distinguish all-gray filenames - added rotating colors to `spack config blame`
2018-10-26docker: unite Dockerfiles; auto-deploy images to DockerHub (#9329)Omar Padron1-10/+7
* Unite Dockerfiles - add build/run/push scripts * update docker documentation * update .travis.yml * switch to using a preprocessor on Dockerfiles * skip building docker images on pull requests * update files with copyright info * tweak when travis builds for docker files are done
2018-10-25Fix autoload of direct dependencies for python (#9630)Massimiliano Culpo3-1/+26
fixes #9624 merge_config_rules was using `strict=False` to check if a spec satisfies a constraint, which loosely translates to "this spec has no conflict with the constraint, so I can potentially add it to the spec". We want instead `strict=True` which means "the spec satisfies the constraint right now".
2018-10-24bugfix: use OS default permissions for mkdirp when mode is not provided (#9604)Todd Gamblin1-4/+13
- #8773 made the default mode 0o777, which is what's documented but mkdirp actually takes the OS default or umask by default - revert to the Python default by default, and only set the mode when asked explicitly.
2018-10-24Fix typo in help message (#9599)Javier Cervantes1-1/+1
2018-10-22Bug fix: Module PATH check (#9574)Satish Balay1-6/+5
#9100 added a warning message when a path extracted from a module file did not appear to be a valid filesystem path. This check was applied to a variable which could be a list of paths, which would erroneously trigger the warning. This commit updates the check to run at the actual point where the path has been extracted.
2018-10-22license: exclude ignored files from the license check (#9578)Todd Gamblin1-1/+1
Previously, if you built the docs, you'd get license errors for generated .rst files. This removes them from the list of licensed files.
2018-10-18Allow setting language of compiler messages (#9486)Michael Kuhn2-0/+8
* Add a build_language config.yaml option which controls the language of compiler messages * build_language defaults to "C", in which case the compiler messages will be in English. This allows Spack log parsing to detect and highlight error messages (since the regular expressions to find error messages are in English) * The user can use the default language in their environment by setting the build_language config variable to null or ''
2018-10-17relicense: add `spack license` commandTodd Gamblin2-0/+224
- `spack license list-files`: list all files that should have license headers - `spack license list-lgpl`: list files still under LGPL-2.1 - `spack license verify`: check that license headers are correct - Added `spack license verify` to style tests
2018-10-17relicense: add Apache2/MIT header to `spack create`Todd Gamblin1-4/+12
2018-10-17relicense: replace LGPL headers with Apache-2.0/MIT SPDX headersTodd Gamblin364-7493/+1532
- remove the old LGPL license headers from all files in Spack - add SPDX headers to all files - core and most packages are (Apache-2.0 OR MIT) - a very small number of remaining packages are LGPL-2.1-only
2018-10-16docs: add note about compilers with modules (#9520)Mark W. Krentel1-0/+8
compilers.yaml can track a module that is needed for a compiler, but Spack does not fill this in automatically. This adds a note to the documentation informing the user how to do this.
2018-10-15meson: explicitly add --libdir (#9504)Michael Kuhn1-0/+6
If we do not specify libdir explicitly, Meson chooses something like lib/x86_64-linux-gnu, which causes problems when trying to find libraries and pkg-config files.
2018-10-12buildcache install: generate modules (#9509)Peter Scheibel1-0/+1
Spack packages installed using spack buildcache were not running post-install hooks, which create module files and manage licenses (if necessary). This was already occurring for Spack packages installed with spack install --use-cache
2018-10-11permissions: add permission configuration to packages.yaml (#8773)Greg Becker9-8/+335
Spack can now be configured to assign permissions to the files installed by a package. In the `packages.yaml` file under `permissions`, the attributes `read`, `write`, and `group` control the package permissions. These attributes can be set per-package, or for all packages under `all`. If permissions are set under `all` and for a specific package, the package-specific settings take precedence. The `read` and `write` attributes take one of `user`, `group`, and `world`. packages: all: permissions: write: group group: spack my_app: permissions: read: group group: my_team
2018-10-10Make CDash reporting easier to use (#9357)Zack Galbreath2-2/+16
* Better default CLI arguments for CDash reporting --log-format=cdash is now implied if you specify the --cdash-upload-url option to spack install. We also now default to writing CTest XML files to cdash_report/ when using the CDash reporter if no --log-file argument was specified. * Improved documentation on how to use the CDash reporter
2018-10-09Make builtin flag handlers available in package scope (#8668)Greg Becker4-38/+45
* Push default flag handlers into module scope * Preserve backwards compatibility of builtin flag handler names Ensure Spack continues to work for packages using the `Package.env_flags` idiom and equivalent. * update docs and tests to match * Update packages to match new syntax
2018-10-02Update Spack on Cray docs for singularity platform=linux workaround (#9398)Greg Becker1-0/+14
2018-09-27Module path parsing: CRAY_LD_LIBRARY_PATH and PACKAGE_DIR (#9374)Peter Scheibel2-3/+24
Fix two bugs with module file parsing: * Detection of the CRAY_LD_LIBRARY_PATH variable was broken by #9100. This fixes it and adds a test for it. * For module names like "foo-bar/1.0", the associated PACKAGE_DIR environment variable name would be "FOO_BAR_DIR", but Spack was not parsing the components and not converting "-" to "_"
2018-09-25Increase and customize lock timeouts (#9219)Peter Scheibel6-89/+204
Fixes #9166 This is intended to reduce errors related to lock timeouts by making the following changes: * Improves error reporting when acquiring a lock fails (addressing #9166) - there is no longer an attempt to release the lock if an acquire fails * By default locks taken on individual packages no longer have a timeout. This allows multiple spack instances to install overlapping dependency DAGs. For debugging purposes, a timeout can be added by setting 'package_lock_timeout' in config.yaml * Reduces the polling frequency when trying to acquire a lock, to reduce impact in the case where NFS is overtaxed. A simple adaptive strategy is implemented, which starts with a polling interval of .1 seconds and quickly increases to .5 seconds (originally it would poll up to 10^5 times per second). A test is added to check the polling interval generation logic. * The timeout for Spack's whole-database lock (e.g. for managing information about installed packages) is increased from 60s to 120s * Users can configure the whole-database lock timeout using the 'db_lock_timout' setting in config.yaml Generally, Spack locks (those created using spack.llnl.util.lock.Lock) now have no timeout by default This does not address implementations of NFS that do not support file locking, or detect cases where services that may be required (nfslock/statd) aren't running. Users may want to be able to more-aggressively release locks when they know they are the only one using their Spack instance, and they encounter lock errors after a crash (e.g. a remote terminal disconnect mentioned in #8915).
2018-09-24macos: add mojave (#9322)Denis Davydov1-1/+2
2018-09-19report error if failed process captures stderr (#9293)scheibelp1-6/+16
When a Spack Executable was configured to capture stderr and the process failed, the error messages of the process were discarded. This made it difficult to understand why the process failed. The exception is now updated to include the stderr of the process when the Executable captures stderr.
2018-09-18Fix version parsing for cistem package (#9260)Adam J. Stewart2-0/+3
Adds 'code' to the list of suffixes that are excluded from version parsing of URLs, such that if a URL contains the string 'cistem-1.0.0-beta-source-code', a version X will substitute in to produce a URL with cistem-X-source-code ('source' was already excluded). The 'cistem' package version is updated to make use of this (and fix a fetching bug with the cistem package). A unit test is added to check this parsing case.
2018-09-12Bug fix: module file path parsing (#9100)scheibelp2-15/+51
Improve Spack's parsing of module show to eliminate some false positives (e.g. accepting MODULEPATH when it is in fact looking for PATH). This makes the following changes: * Updates the pattern searching for several paths to avoid the case where they are prefixes of unwanted paths * Adds a warning message when an extracted path doesn't exist (which may help catch future module parsing bugs faster) * Adds a test with the content mentioned in #9083