summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2022-10-13meson: remove slash in path (#33292)Adam J. Stewart1-1/+1
2022-10-13gitlab ci: Print better information about broken specs (#33124)Scott Wittenburg3-49/+98
When a pipeline generation job is automatically failed because it generated jobs for specs known to be broken on develop, print better information about the broken specs that were encountered. Include at a minimum the hash and the url of the job whose failure caused it to be put on the broken specs list in the first place.
2022-10-13Add missing upperbound to docs/spack.yaml (#33280)Harmen Stoppels1-0/+2
2022-10-12env depfile: allow deps only install (#33245)Harmen Stoppels2-65/+32
* env depfile: allow deps only install - Refactor `spack env depfile` to use a Jinja template, making it a bit easier to follow as a human being. - Add a layer of indirection in the generated Makefile through an `<prefix>/.install-deps/<hash>` target, which allows one to specify different options when installing dependencies. For example, only verbose/debug mode on when installing some particular spec: ``` $ spack -e my_env env depfile -o Makefile --make-target-prefix example $ make example/.install-deps/<hash> -j16 $ make example/.install/<hash> SPACK="spack -d" SPACK_INSTALL_FLAGS=--verbose -j16 ``` This could be used to speed up `spack ci rebuild`: - Parallel install of dependencies from buildcache - Better readability of logs, e.g. reducing verbosity when installing dependencies, and splitting logs into deps.log and current_spec.log * Silence please!
2022-10-12Fix typo in docs (#33182)Massimiliano Culpo1-1/+1
2022-10-11Specify GCC prefix in LLVM-based compilers (#33146)Harmen Stoppels2-0/+29
* spack.compiler.Compiler: introduce prefix property We currently don't really have something that gives the GCC install path, which is used by many LLVM-based compilers (llvm, llvm-amdgpu, nvhpc, ...) to fix the GCC toolchain once and for all. This `prefix` property is dynamic in the sense that it queries the compiler itself. This is necessary because it's not easy to deduce the install path from the `cc` property (might be a symlink, might be a filename like `gcc` which works by having the compiler load a module that sets the PATH variable, might be a generic compiler wrapper based on environment variables like on cray...). With this property introduced, we can clean up some recipes that have the logic repeated for GCC. * intel-oneapi-compilers: set --gcc-sysroot to %gcc prefix
2022-10-11Refactor a few classes related to package repositories (#32273)Massimiliano Culpo27-669/+466
Caches used by repositories don't reference the global spack.repo.path instance anymore, but get the repository they refer to during initialization. Spec.virtual now use the index, and computation done to compute the index use Repository.is_virtual_safe. Code to construct mock packages and mock repository has been factored into a unique MockRepositoryBuilder that is used throughout the codebase. Add debug print for pushing and popping config scopes. Changed spack.repo.use_repositories so that it can override or not previous repos spack.repo.use_repositories updates spack.config.config according to the modifications done Removed a peculiar behavior from spack.config.Configuration where push would always bubble-up a scope named command_line if it existed
2022-10-11`spack diff` any specs you want (#32737)Peter Scheibel1-4/+6
Resolves #31782 With this change, if a spec is concrete after parsing (e.g. spec.yaml or /hash-based), then it is not disambiguated (a process which requires (a) that the spec be installed and (b) that it be part of the currently-active environment). This commit allows you to: * Diff specs from an environment regardless of whether they have been installed (more useful for projection/matrix-based envs) * Diff specs read from .yaml files which may or may not be entirely different installations of Spack Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-10-10Docs: Getting Started Dependencies (#32480)Axel Huebl4-2/+33
* Docs: Getting Started Dependencies Finally document what one needs to install to use Spack on Linux and Mac :-) With <3 for minimal container users and my colleagues with their fancy Macs. * Debian Update Packages: GCC, Python - build-essential: includes gcc, g++ (thx Cory) - Python: add python3-venv, python3-distutils (thx Pradyun) * Add RHEL8 Dependencies
2022-10-10shell prompt: enclose control sequence in brackets (#33079)Brian Vanderwende2-4/+10
When setting `PS1` in Bash, it's required to enclose non-printable characters in square brackets, so that the width of the terminal is handled correctly. See https://www.gnu.org/software/bash/manual/bash.html#Controlling-the-Prompt
2022-10-10PythonPackage: fix libs/headers attributes (#32970)Adam J. Stewart1-3/+3
2022-10-07autotools: extend patching of the libtool script (#30768)Sergey Kosukhin4-34/+208
* filter_file: introduce argument 'start_at' * autotools: extend patching of the libtool script * autotools: refactor _patch_usr_bin_file * autotools: improve readability of the filtering * autotools: keep the modification time of the configure scripts * autotools: do not try to patch directories * autotools: explain libtool patching for posterity Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com> Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-10-07add compatibility for rocky8 and rhel8 (#33068)MatthewLieber1-0/+3
2022-10-06Add a warning on Python 2.7 deprecation (#33052)Massimiliano Culpo2-0/+9
Co-authored-by: alalazo <alalazo@users.noreply.github.com> Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2022-10-06tcl module template: automatically unload automatically loaded modules (#32853)Sergey Kosukhin1-1/+1
Remove `module-info mode load` condition that prevents auto-unloading when autoloading is enabled. It looks like this condition was added to work around an issue in environment-modules that is no longer necessary. Add quotes to make is-loaded happy
2022-10-06docs: fix deprecated use of install_tree (#33004)Harmen Stoppels1-3/+3
2022-10-06cuda: add v11.8.0 (#33027)Andrew W Elble1-1/+10
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
2022-10-05Optimize concurrent misc_cache provider index rebuild (#32874)Jordan Galby1-2/+10
When concurrent misc_cache provider index rebuilds happen, try to rebuild it only once, so we don't exceed misc_cache lock timeout. For example, when using `spack env depfile`, with no previous misc_cache, running `make -f depfile -j8` could run at most 8 concurrent `spack install` locking on misc_cache to rebuild the provider index. If one rebuild takes 30s, before this fix, the "worst" lock could wait up to 30s * 7, easily exceeding misc_cache lock timeout. Now, the "worst" lock would take 30s * 1 + ~1s * 6.
2022-10-05autotools: Filter libtools when building with dpcpp (#32876)Chris MacMackin1-1/+9
Due to a [known issue](https://community.intel.com/t5/Intel-oneAPI-Data-Parallel-C/dpcpp-and-GNU-Autotools/m-p/1296985) with dpcpp, autotool-based builds that try to use it will fail because they try to link against temporary files that no longer exist. This commit filters those files out of the libtools script so that linking can work properly.
2022-10-04BuildEnvironment: accumulate module changes to poke to all relevant modules ↵Greg Becker2-1/+26
(#32340) Currently, module changes from `setup_dependent_package` are applied only to the module of the package class, but not to any parent classes' modules between the package class module and `spack.package_base`. In this PR, we create a custom class to accumulate module changes, and apply those changes to each class that requires it. This design allows us to code for a single module, while applying the changes to multiple modules as needed under the hood, without requiring the user to reason about package inheritance.
2022-10-04`find`/`list`: display package counts last (#32946)Todd Gamblin5-6/+53
* find/list: display package counts last We have over 6,600 packages now, and `spack list` still displays the number of packages before it lists them all. This is useless for large sets of results (e.g., with no args) as the number has scrolled way off the screen before you can see it. The same is true for `spack find` with large installations. This PR changes `spack find` and `spack list` so that they display the package count last. * add some quick testing Co-authored-by: Danny McClanahan <1305167+cosmicexplorer@users.noreply.github.com>
2022-10-04Fix typo in documentation (#32984)Adam J. Stewart1-1/+1
2022-10-02Fixup errors introduced by Clingo Pr: (#32905)John W. Parent1-1/+1
* re2c depends on cmake on Windows * Winbison properly added to bootstrap package search list
2022-09-30develop: canonicalize dev paths and base relative paths on `env.path` (#30075)Greg Becker6-12/+60
Allow environment variables and spack-specific path substitution variables (e.g. `$spack`) to be used in the paths associated with develop specs, while maintaining the ability to keep those paths relative to the environment rather than the working directory.
2022-09-29Install: Add use-buildcache option to install (#32537)kwryankrattiger3-8/+166
Install: Add use-buildcache option to install * Allow differentiating between top level packages and dependencies when determining whether to install from the cache or not. * Add unit test for --use-buildcache * Use metavar to display use-buildcache options. * Update spack-completion
2022-09-28Windows: Support for Clingo and dependencies (#30690)John W. Parent4-5/+14
Make it possible to install the Clingo package on Windows; this also provides a means to use Clingo with Spack on Windows. This includes * A new "winbison" package: Windows has a port of bison and flex where the two packages are grouped together. Clingo dependencies have been updated to use winbison on Windows and bison elsewhere (this avoids complicating the existin bison/flex packages until we can add support for implied virtuals). * The CMake build system was incorrectly converting CMAKE_INSTALL_PREFIX to POSIX format. * The re2c package has been modified to use CMake on Windows; for now this is done by overloading the configure/build/install methods to perform CMake-appropriate operations; the package should be refactored once support for multiple build systems in one Package is available.
2022-09-28Remove mentions of "best-effort" matrix expansion in the docs (#32755)Massimiliano Culpo1-25/+0
closes #20340
2022-09-27shell.py: fix undefined variable in csh deactivate (#32816)Alex Hornburg1-2/+2
This commit fixes #27027. The root cause of the issue is that the `SPACK_OLD_PROMPT` variable was evaluated in string interpolation regardless of whether the guard condition above evaluates to true or false. This commit uses the `eval` keyword to defer evaluation until the command is executed. Co-authored-by: Alexander Hornburg <alexande@xilinx.com>
2022-09-26improve lexing error (#32832)Chris White1-1/+3
2022-09-26Fetching/decompressing: use magic numbers (#31589)John W. Parent9-137/+524
Spack currently depends on parsing filenames of downloaded files to determine what type of archive they are and how to decompress them. This commit adds a preliminary check based on magic numbers to determine archive type (but falls back on name parsing if the extension type cannot be determined). As part of this work, this commit also enables decompression of .tar.xz-compressed archives on Windows.
2022-09-23Fix two docstring typos (#32751)Tamara Dahlgren1-2/+2
2022-09-23Skip all tests using problematic fixture on python 2.7 (#32768)Peter Scheibel1-0/+6
2022-09-22Fetching: log more exceptions (#32736)Peter Scheibel2-1/+3
Include exception info related to url retrieval in debug messages which otherwise would be swallowed. This is intended to be useful for detecting if CA configuration interferes with downloads from HTTPS links.
2022-09-22Add hash hint to multi-spec message (#32652)Tamara Dahlgren3-25/+22
2022-09-21Allow conditional variants as first values in a variant directive (#32740)Massimiliano Culpo2-3/+10
2022-09-21Docs: Update pipeline ci rebuild to add --tests (plus fixed typos) (#32048)Tamara Dahlgren1-24/+64
2022-09-17Add skip_import to PythonPackage and use it in py-nilearn (#32664)Manuela Kuhn2-1/+30
* Add skip_import to PythonPackage and use it in py-nilearn * Fix dependencies
2022-09-16bugfix: package hash should affect process, dag, and dunder hashes (#32516)Todd Gamblin5-1/+40
This fixes a bug where two installations that differ only by package hash will not show up in `spack find`. The bug arose because `_cmp_node` on `Spec` didn't include the package hash in its yielded fields. So, any two `Spec` objects that were only different by package hash would appear to be equal and would overwrite each other when inserted into the same `dict`. Note that we could still *install* specs with different package hashes, and they would appear in the database, but we code that needed to put them into data structures that use `__hash__` would have issues. This PR makes `Spec.__hash__` and `Spec.__eq__` include the `process_hash()`, and it makes `Spec._cmp_node` include the package hash. All of these *should* include all information in a spec so that we don't end up in a situation where we are blind to particular field differences. Eventually, we should unify the `_cmp_*` methods with `to_node_dict` so there aren't two sources of truth, but this needs some thought, since the `_cmp_*` methods exist for speed. We should benchmark whether it's really worth having two types of hashing now that we use `json` instead of `yaml` for spec hashing. - [x] Add `package_hash` to `Spec._cmp_node` - [x] Add `package_hash` to `spack.solve.asp.spec_clauses` so that the `package_hash` will show up in `spack diff`. - [x] Add `package_hash` to the `process_hash` (which doesn't affect abstract specs but will make concrete specs correct) - [x] Make `_cmp_iter` report the dag_hash so that no two specs with different process hashes will be considered equal.
2022-09-14Bugfix: find_libraries (#32653)John W. Parent2-9/+32
53a7b49 created a reference error which broke `.libs` (and `find_libraries`) for many packages. This fixes the reference error and improves the testing for `find_libraries` by actually checking the extension types of libraries that are retrieved by the function.
2022-09-14Manifest parsing: skip invalid manifest files (#32467)Peter Scheibel3-4/+57
* catch json schema errors and reraise as property of SpackError * no need to catch subclass of given error * Builtin json library for Python 2 uses more generic type * Correct instantiation of SpackError (requires a string rather than an exception) * Use exception chaining (where possible)
2022-09-14Allow version to accept the '=' token without activating lexer switch (#32257)psakievich2-54/+126
2022-09-14Update module_file_support.rst (#32629)Sam Broderick1-1/+1
Missing lead in "This may to inconsistent modulfiles if..."
2022-09-13Windows rpath support (#31930)John W. Parent14-40/+271
Add a post-install step which runs (only) on Windows to modify an install prefix, adding symlinks to all dependency libraries. Windows does not have the same concept of RPATHs as Linux, but when resolving symbols will check the local directory for dependency libraries; by placing a symlink to each dependency library in the directory with the library that needs it, the package can then use all Spack-built dependencies. Note: * This collects dependency libraries based on Package.rpath, which includes only direct link dependencies * There is no examination of libraries to check what dependencies they require, so all libraries of dependencies are symlinked into any directory of the package which contains libraries
2022-09-13Added install phase test methods for CachedCMakePackage (inherited) and ↵Tamara Dahlgren1-0/+6
WafPackage (#32627)
2022-09-10ci: restore coverage computation (#32585)Massimiliano Culpo2-1/+5
* ci: restore coverage computation * Mark "test_foreground_background" as xfail * Mark "test_foreground_background_output" as xfail * Make number of processes explicit, remove verbosity on linux * Run coverage on just 3 Python jobs for linux * Run coverage on just 3 Python jobs for linux * Run coverage on just 2 Python jobs for linux * Add back verbose, since before we didn't encounter the xdist internal error * Reduce the workers to 2 * Try to use command line
2022-09-09GitRef Versions: Bug Fixes (#32437)psakievich4-3/+47
* Fix a version cmp bug in asp.py * Fix submodule bug for git refs * Add branch in logic for submodules * Fix git version comparisons main does not satisfy git.foo=main git.foo=main does satisfy main
2022-09-08CMakePackage: allow custom CMAKE_PREFIX_PATH entries (#32547)Peter Scheibel3-2/+17
* define `cmake_prefix_paths` property for packages add to CMake arguments via CMakePackage Co-authored-by: Robert Underwood <runderwood@anl.gov>
2022-09-08Skip test which fails randomly on Python 2.7 (#32569)Peter Scheibel1-0/+3
2022-09-07Make GHA tests parallel by using xdist (#32361)Tom Scogland14-79/+114
* Add two no-op jobs named "all-prechecks" and "all" These are a suggestion from @tgamblin, they are stable named markers we can use from gitlab and possibly for required checks to make CI more resilient to refactors changing the names of specific checks. * Enable parallel testing using xdist for unit testing in CI * Normalize tmp paths to deal with macos * add -u flag compatibility to spack python As of now, it is accepted and ignored. The usage with xdist, where it is invoked specifically by `python -u spack python` which is then passed `-u` by xdist is the entire reason for doing this. It should never be used without explicitly passing -u to the executing python interpreter. * use spack python in xdist to support python 2 When running on python2, spack has many import cycles unless started through main. To allow that, this uses `spack python` as the interpreter, leveraging the `-u` support so xdist doesn't error out when it unconditionally requests unbuffered binary IO. * Use shutil.move to account for tmpdir being in a separate filesystem sometimes
2022-09-06Fix spack locking on some NFS systems (#32426)Seth R. Johnson1-2/+6
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>