summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2022-02-21Bring back cuda@11.4.0 conflicts for GCC and clang; add 11.4.3:11.4.4 (#29076)Alberto Invernizzi1-0/+2
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-02-18spack external find: change default behavior (#29031)Massimiliano Culpo2-4/+22
See https://github.com/spack/spack/issues/25353#issuecomment-1041868116 This commit changes the default behavior of ``` $ spack external find ``` from searching all the possible packages Spack knows about to search only for the ones tagged as being a "build-tool". It also introduces a `--all` option to restore the old behavior.
2022-02-17darwin: robust macos version detection (#28991)Seth R. Johnson2-7/+49
Prefer `sw_vers` to `platform.mac_ver`. In anaconda3 installation, for example, the latter reports 10.16 on Monterey -- I think this is affected by how and where the python instance was built. Use MACOSX_DEPLOYMENT_TARGET if present to override the operating system choice.
2022-02-17gitlab: Propagate stack name to downstream build jobs (#29019)Scott Wittenburg1-0/+4
It will be useful for metrics gathering and possibly debugging to have this environment variable available in the runner pods that do the actual rebuilds.
2022-02-17Testing: optionally run tests on externally installed packages (#28701)Tamara Dahlgren5-8/+64
Since Spack does not install external packages, this commit skips them by default when running stand-alone tests. The assumption is that such packages have likely undergone an acceptance test process. However, the tests can be run against installed externals using ``` % spack test run --externals ... ```
2022-02-17Reduce verbosity of patches=... variant (#29015)Harmen Stoppels1-2/+19
* reduce verbosity of patches=... variant * Special-case prefix-matches for satisfies of patches variant
2022-02-17ASP-based solver: don't sort when defining variant possible values (#29013)Massimiliano Culpo1-2/+3
fixes #28260 Since we iterate over different variants from many packages, the variant values may have types which are not comparable, which causes errors at runtime. This is not a real issue though, since we don't need the facts to be ordered. Thus, to avoid needless sorting, the sorted function has been removed and a comment has been added to tip any developer that might need to inspect these clauses for debugging to add back sorting on the first two items only. It's kind of difficult to add a test for this, since the error depends on whether Python sorting algorithm ever needs to compare the third value of a tuple being ordered.
2022-02-16extensions: allow multiple "extends" directives (#28853)Tom Scogland2-5/+28
* extensions: allow multiple "extends" directives This will allow multiple extends directives in a package as long as only one of them is selected as a dependency in the concrete spec. * document the option to have multiple extends
2022-02-16concretizer: refactor argument passing for reuseTodd Gamblin6-107/+106
Reuse previously was a very invasive change that required parameters to be added to all the methods that called `concretize()` on a `Spec` object. With the addition of concretizer configuration, we can use the config system to simplify this argument passing and keep the code cleaner. We decided that concretizer config options should be read at `Solver` instantiation time, and if config changes between instnatiation of a particular solver and `solve()` invocation, the `Solver` should use the settings from `__init__()`. - [x] remove `reuse` keyword argument from most concretize functions - [x] refactor usages to use `spack.config.override("concretizer:reuse", True)` - [x] rework argument passing in `Solver` so that parameters are set from config at instantiation time
2022-02-16docs: add section on concretizer configurationTodd Gamblin5-13/+59
* Document `concretizer.yaml`, `--reuse`, and `--fresh`.
2022-02-16commands: refactor `--reuse` handling to use configTodd Gamblin11-35/+135
`--reuse` was previously handled individually by each command that needed it. We are growing more concretization options, and they'll need their own section for commands that support them. Now there are two concretization options: * `--reuse`: Attempt to reuse packages from installs and buildcaches. * `--fresh`: Opposite of reuse -- traditional spack install. To handle thes, this PR adds a `ConfigSetAction` for `argparse`, so that you can write argparse code like this: ``` subgroup.add_argument( '--reuse', action=ConfigSetAction, dest="concretizer:reuse", const=True, default=None, help='reuse installed dependencies/buildcaches when possible' ) ``` With this, you don't need to add logic to pull the argument out and handle it; the `ConfigSetAction` just does it for you. This can probably be used to clean up some other commands later, as well. Code that was previously passing `reuse=True` around everywhere has been refactored to use config, and config is set from the CLI using a new `add_concretizer_args()` function in `spack.cmd.common.arguments`. - [x] Add `ConfigSetAction` to simplify concretizer config on the CLI - [x] Refactor code so that it does not pass `reuse=True` to every function. - [x] Refactor commands to use `add_concretizer_args()` and to pass concretizer config using the config system.
2022-02-16tests: consolidate mock scope creation logic in `conftest.py`Todd Gamblin2-22/+18
Config scopes were different for `config` and `mutable_config`, and `mutable_config` did not have a command line scope. - [x] Fix by consolidating the creation logic for the two fixtures.
2022-02-16config: add a new `concretizer` config sectionTodd Gamblin3-2/+41
The concretizer is going to grow to have many more configuration, and we really need some structured config for that. * We have the `config:concretizer` option that chooses the solver, but extending that is awkward (we'd need to replace a string with a `dict`) and the solver choice will be deprecated eventually. * We have the `concretization` option in environments, but it's not a top-level config section -- it's just for environments, and it also only admits a string right now. To avoid overlapping with either of these and to allow the most extensibility in the future, this adds a new `concretizer` config section that can be used in and outside of environments. There is only one option right now: `reuse`. This can expand to include other options later. Likely, we will soon deprecate `config:concretizer` and warn when the user doesn't use `clingo`, and we will eventually (sometime later) move the `together` / `separately` options from `concretization` into the top-level `concretizer` section. This commit just adds the new section and schema. Fully wiring it up is TBD.
2022-02-16refactor: convert spack.solver.asp.solve() to a classTodd Gamblin4-46/+95
The solver has a lot of configuration associated with it. Rather than adding arguments to everything, we should encapsulate that in a class. This is the start of that work; it replaces `solve()` and its kwargs with a class and properties.
2022-02-16Add 'stable' to the list of infinity version names. (#28772)Mark W. Krentel2-2/+3
* Add 'stable' to the list of infinity version names. Rename libunwind 1.5-head to 1.5-stable. * Add stable to the infinite version list in packaging_guide.rst.
2022-02-16find_libraries: search for both .so and .dylib on macOS (#28924)Adam J. Stewart1-4/+10
2022-02-15intel compiler: fix link time error with `LLVMgold.so` (#28731)Stephen Sachs1-0/+38
The Intel compiler will, at link time, call `ld -plugin LLVMgold.so`, which expects libraries like `libimfo.so` to be found either in the `LD_LIBRARY_PATH` or in `LLVMgold.so`s RPATH. As `LLVMgold.so` already uses RUNPATH, I used that to extend this to the necessary library locations. This PR should fix issues: https://github.com/spack/spack/issues/10308 https://github.com/spack/spack/issues/18606 https://github.com/spack/spack/issues/17100 https://github.com/spack/spack/issues/21237 https://github.com/spack/spack/issues/4261 Co-authored-by: Stephen Sachs <stesachs@amazon.com>
2022-02-15Make `spack -e [env] spec` show environment root specs (#25941)Harmen Stoppels2-19/+47
2022-02-15Unalias despacktivate only when alias exists (#28939)Harmen Stoppels1-1/+1
2022-02-15darwin: make sure MACOSX_DEPLOYMENT_TARGET has a minor component (#28926)Seth R. Johnson1-1/+10
2022-02-11move typing_extensions.py back into typing.py =\ (#28549)Danny McClanahan3-29/+26
2022-02-10macOS: always set `MACOSX_DEPLOYMENT_TARGET` (#28797)Seth R. Johnson5-6/+75
* core: Make platform environment an instance not class method In preparation for accessing data constructed in __init__. * macos: set consistent macosx deployment target This should silence numerous warnings from mixed gcc/macos toolchains. * perl: prevent too-new deployment target version ``` *** Unexpected MACOSX_DEPLOYMENT_TARGET=11 *** *** Please either set it to a valid macOS version number (e.g., 10.15) or to empty. ``` * Stylin' * Add deployment target overrides to failing autoconf packages * Move configure workaround to base autoconf package This reverts commit 3c119eaf8b4fb37c943d503beacf5ad2aa513d4c. * Stylin' * macos: add utility functions for SDK These aren't yet used but should probably be added to spack debug report.
2022-02-10ASP-based solver: reduce input facts and add heuristic (#28848)Massimiliano Culpo2-44/+49
* Remove node_target_satisfies/3 in favor of target_satisfies/2 When emitting input facts we don't need to couple target with packages, but we can emit fewer facts independently and let the grounder combine them. * Remove compiler_version_satisfies/4 in favor of compiler_version_satisfies/3 When emitting input facts we don't need to couple compilers with packages, but we can emit fewer facts independently and let the grounder combine them. * Introduce heuristic in the ASP-program With heuristic we can drive clingo to make better initial guesses, which lead to fewer choices and conflicts in the overall solve
2022-02-10Fix CMakePackage.define for libs/headers (#28838)Seth R. Johnson2-1/+7
The 'libs' property returned by a spec is not a list nor tuple. Closes #28836.
2022-02-10spack audit: fix spurious failures for target/platform conflicts (#28860)Greg Becker2-5/+15
2022-02-09Tests: move has_test_method to spack.package (#28813)Tamara Dahlgren4-22/+20
2022-02-09Detecting "Cray" as "linux" during bootstrap (#28726)Massimiliano Culpo3-34/+47
* bootstrap: avoid detecting "Cray" and treat the platform as "linux" * bootstrap: create a proper context manager to disable cray
2022-02-07intel-mkl: BLACS with intel-oneapi-mpi (#28476)Stephen Sachs1-0/+1
Identify the correct BLACS libaries when `intel-oneapi-mpi` is used.
2022-02-07Support config variables in config.yaml extensions paths (#17772)Jordan Galby4-10/+32
2022-02-04database: fix reindex with uninstalled deps (#28764)Harmen Stoppels4-42/+94
* Fix reindex with uninstalled deps When a prefix of a dep is removed, and the db is reindexed, it is added through the dependent, but until now it incorrectly listed the spec as 'installed'. There was also some questionable behavior in the db when the same spec was added multiple times, it would always be marked installed. * Always reserve path * Only add installed spec's prefixes to install prefixes set * Improve warning, and ensure ensure only ensures * test: reindex with every file system remnant removed except for the old index; it should give a database with nothing installed, including records with installed==False,external==False,ref_count==0,explicit=True, and these should be removable from the database
2022-02-04Use Spec.constrain to construct spec lists for stacks (#28783)Massimiliano Culpo4-40/+62
* stacks: add regression tests for matrix expansion * Use constrain semantics to construct spec lists for stacks * Fix semantics for constraining an anonymous spec. Add tests
2022-02-02Add a "sticky" property to variants (#28630)Massimiliano Culpo8-101/+176
* Add sticky variants * Add unit tests for sticky variants * Add documentation for sticky variants * Revert "Revert 19736 because conflicts are avoided by clingo by default (#26721)" This reverts commit 33ef7d57c1a09d6ce454503e9ff5638747e4f4a2. * Add stickiness to "allow-unsupported-compiler"
2022-02-01TermStatusLine: fix python 2.7 and add test (#28715)Harmen Stoppels2-1/+12
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-01-28macholib, altgraph: update vendored dependency (#28664)Massimiliano Culpo25-1253/+1345
2022-01-26Ensure "spack unit-test" can bootstrap clingo (#28572)Massimiliano Culpo2-1/+8
2022-01-26Print 'Waiting for another process to install x, y, z' in distributed builds ↵Harmen Stoppels1-0/+53
(#28535) Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-01-24Resource stage: no space before colon (#28560)Adam J. Stewart1-2/+2
2022-01-23Fix spack -C command_line_scope with other flags (#28418)Jordan Galby1-3/+3
The option `spack -C` was broken if another flag that touched config was also set (`spack -C ... -d`, `-c config:...` etc...).
2022-01-21introduce `llnl.util.compat` to remove sys.version_info checks (#21720)Danny McClanahan31-246/+247
- also split typing.py into typing_extensions and add py2 shims
2022-01-21CUDA: add v11.6.0 (#28439)Bram Veenboer1-4/+3
2022-01-21add six.raise_from() to preserve exception traceback (#28532)Danny McClanahan10-22/+213
* add six.raise_from() to preserve exception traceback * add tests for code coverage
2022-01-19SIPPackage: fix build system (#28447)Adam J. Stewart1-6/+2
2022-01-19spack compiler info exit 1 when no compilers are found (#28493)Logan Harbour1-1/+1
fixes #28492
2022-01-18Switch lmod default all:autoload from none to direct (#28357)Harmen Stoppels34-40/+70
* Switch lmod module all autoload default from none to direct * Fix the docs
2022-01-17is_system_path: return False if path is None (#28403)Tamara Dahlgren2-1/+3
2022-01-14Update copyright year to 2022Todd Gamblin542-546/+546
2022-01-14`spack license update-copyright-year` now updates the `LICENSE-MIT` fileTodd Gamblin2-10/+36
`spack license update-copyright-year` was updating license headers but not the MIT license file. Make it do that and add a test. Also simplify the way we bump the latest copyright year so that we only need to update it in one place.
2022-01-14PythonPackage: install packages with pip (#27798)Adam J. Stewart6-719/+391
* Use pip to bootstrap pip * Bootstrap wheel from source * Update PythonPackage to install using pip * Update several packages * Add wheel as base class dep * Build phase no longer exists * Add py-poetry package, fix py-flit-core bootstrapping * Fix isort build * Clean up many more packages * Remove unused import * Fix unit tests * Don't directly run setup.py * Typo fix * Remove unused imports * Fix issues caught by CI * Remove custom setup.py file handling * Use PythonPackage for installing wheels * Remove custom phases in PythonPackages * Remove <phase>_args methods * Remove unused import * Fix various packages * Try to test Python packages directly in CI * Actually run the pipeline * Fix more packages * Fix mappings, fix packages * Fix dep version * Work around bug in concretizer * Various concretization fixes * Fix gitlab yaml, packages * Fix typo in gitlab yaml * Skip more packages that fail to concretize * Fix? jupyter ecosystem concretization issues * Solve Jupyter concretization issues * Prevent duplicate entries in PYTHONPATH * Skip fenics-dolfinx * Build fewer Python packages * Fix missing npm dep * Specify image * More package fixes * Add backends for every from-source package * Fix version arg * Remove GitLab CI stuff, add py-installer package * Remove test deps, re-add install_options * Function declaration syntax fix * More build fixes * Update spack create template * Update PythonPackage documentation * Fix documentation build * Fix unit tests * Remove pip flag added only in newer pip * flux: add explicit dependency on jsonschema * Update packages that have been added since this was branched off of develop * Move Python 2 deprecation to a separate PR * py-neurolab: add build dep on py-setuptools * Use wheels for pip/wheel * Allow use of pre-installed pip for external Python * pip -> python -m pip * Use python -m pip for all packages * Fix py-wrapt * Add both platlib and purelib to PYTHONPATH * py-pyyaml: setuptools is needed for all versions * py-pyyaml: link flags aren't needed * Appease spack audit packages * Some build backend is required for all versions, distutils -> setuptools * Correctly handle different setup.py filename * Use wheels for py-tomli to avoid circular dep on py-flit-core * Fix busco installation procedure * Clarify things in spack create template * Test other Python build backends * Undo changes to busco * Various fixes * Don't test other backends
2022-01-13Python: improve site_packages_dir handling (#28346)Adam J. Stewart3-33/+27
* Python: improve site_packages_dir handling * Replace all site_packages_dir with purelib/platlib
2022-01-12Do not initialize config on spack compiler list (#28042)Harmen Stoppels4-11/+37
When `spack compiler list` is run without being restricted to a particular scope, and no compilers are found, say that none are available, and hint that the use should run spack compiler find to auto detect compilers. * Improve docs * Check if stdin is a tty * add a test