summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2022-02-11move typing_extensions.py back into typing.py =\ (#28549)Danny McClanahan3-29/+26
2022-02-10macOS: always set `MACOSX_DEPLOYMENT_TARGET` (#28797)Seth R. Johnson5-6/+75
* core: Make platform environment an instance not class method In preparation for accessing data constructed in __init__. * macos: set consistent macosx deployment target This should silence numerous warnings from mixed gcc/macos toolchains. * perl: prevent too-new deployment target version ``` *** Unexpected MACOSX_DEPLOYMENT_TARGET=11 *** *** Please either set it to a valid macOS version number (e.g., 10.15) or to empty. ``` * Stylin' * Add deployment target overrides to failing autoconf packages * Move configure workaround to base autoconf package This reverts commit 3c119eaf8b4fb37c943d503beacf5ad2aa513d4c. * Stylin' * macos: add utility functions for SDK These aren't yet used but should probably be added to spack debug report.
2022-02-10ASP-based solver: reduce input facts and add heuristic (#28848)Massimiliano Culpo2-44/+49
* Remove node_target_satisfies/3 in favor of target_satisfies/2 When emitting input facts we don't need to couple target with packages, but we can emit fewer facts independently and let the grounder combine them. * Remove compiler_version_satisfies/4 in favor of compiler_version_satisfies/3 When emitting input facts we don't need to couple compilers with packages, but we can emit fewer facts independently and let the grounder combine them. * Introduce heuristic in the ASP-program With heuristic we can drive clingo to make better initial guesses, which lead to fewer choices and conflicts in the overall solve
2022-02-10Fix CMakePackage.define for libs/headers (#28838)Seth R. Johnson2-1/+7
The 'libs' property returned by a spec is not a list nor tuple. Closes #28836.
2022-02-10spack audit: fix spurious failures for target/platform conflicts (#28860)Greg Becker2-5/+15
2022-02-09Tests: move has_test_method to spack.package (#28813)Tamara Dahlgren4-22/+20
2022-02-09Detecting "Cray" as "linux" during bootstrap (#28726)Massimiliano Culpo3-34/+47
* bootstrap: avoid detecting "Cray" and treat the platform as "linux" * bootstrap: create a proper context manager to disable cray
2022-02-07intel-mkl: BLACS with intel-oneapi-mpi (#28476)Stephen Sachs1-0/+1
Identify the correct BLACS libaries when `intel-oneapi-mpi` is used.
2022-02-07Support config variables in config.yaml extensions paths (#17772)Jordan Galby4-10/+32
2022-02-04database: fix reindex with uninstalled deps (#28764)Harmen Stoppels4-42/+94
* Fix reindex with uninstalled deps When a prefix of a dep is removed, and the db is reindexed, it is added through the dependent, but until now it incorrectly listed the spec as 'installed'. There was also some questionable behavior in the db when the same spec was added multiple times, it would always be marked installed. * Always reserve path * Only add installed spec's prefixes to install prefixes set * Improve warning, and ensure ensure only ensures * test: reindex with every file system remnant removed except for the old index; it should give a database with nothing installed, including records with installed==False,external==False,ref_count==0,explicit=True, and these should be removable from the database
2022-02-04Use Spec.constrain to construct spec lists for stacks (#28783)Massimiliano Culpo4-40/+62
* stacks: add regression tests for matrix expansion * Use constrain semantics to construct spec lists for stacks * Fix semantics for constraining an anonymous spec. Add tests
2022-02-02Add a "sticky" property to variants (#28630)Massimiliano Culpo8-101/+176
* Add sticky variants * Add unit tests for sticky variants * Add documentation for sticky variants * Revert "Revert 19736 because conflicts are avoided by clingo by default (#26721)" This reverts commit 33ef7d57c1a09d6ce454503e9ff5638747e4f4a2. * Add stickiness to "allow-unsupported-compiler"
2022-02-01TermStatusLine: fix python 2.7 and add test (#28715)Harmen Stoppels2-1/+12
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-01-28macholib, altgraph: update vendored dependency (#28664)Massimiliano Culpo25-1253/+1345
2022-01-26Ensure "spack unit-test" can bootstrap clingo (#28572)Massimiliano Culpo2-1/+8
2022-01-26Print 'Waiting for another process to install x, y, z' in distributed builds ↵Harmen Stoppels1-0/+53
(#28535) Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-01-24Resource stage: no space before colon (#28560)Adam J. Stewart1-2/+2
2022-01-23Fix spack -C command_line_scope with other flags (#28418)Jordan Galby1-3/+3
The option `spack -C` was broken if another flag that touched config was also set (`spack -C ... -d`, `-c config:...` etc...).
2022-01-21introduce `llnl.util.compat` to remove sys.version_info checks (#21720)Danny McClanahan31-246/+247
- also split typing.py into typing_extensions and add py2 shims
2022-01-21CUDA: add v11.6.0 (#28439)Bram Veenboer1-4/+3
2022-01-21add six.raise_from() to preserve exception traceback (#28532)Danny McClanahan10-22/+213
* add six.raise_from() to preserve exception traceback * add tests for code coverage
2022-01-19SIPPackage: fix build system (#28447)Adam J. Stewart1-6/+2
2022-01-19spack compiler info exit 1 when no compilers are found (#28493)Logan Harbour1-1/+1
fixes #28492
2022-01-18Switch lmod default all:autoload from none to direct (#28357)Harmen Stoppels34-40/+70
* Switch lmod module all autoload default from none to direct * Fix the docs
2022-01-17is_system_path: return False if path is None (#28403)Tamara Dahlgren2-1/+3
2022-01-14Update copyright year to 2022Todd Gamblin542-546/+546
2022-01-14`spack license update-copyright-year` now updates the `LICENSE-MIT` fileTodd Gamblin2-10/+36
`spack license update-copyright-year` was updating license headers but not the MIT license file. Make it do that and add a test. Also simplify the way we bump the latest copyright year so that we only need to update it in one place.
2022-01-14PythonPackage: install packages with pip (#27798)Adam J. Stewart6-719/+391
* Use pip to bootstrap pip * Bootstrap wheel from source * Update PythonPackage to install using pip * Update several packages * Add wheel as base class dep * Build phase no longer exists * Add py-poetry package, fix py-flit-core bootstrapping * Fix isort build * Clean up many more packages * Remove unused import * Fix unit tests * Don't directly run setup.py * Typo fix * Remove unused imports * Fix issues caught by CI * Remove custom setup.py file handling * Use PythonPackage for installing wheels * Remove custom phases in PythonPackages * Remove <phase>_args methods * Remove unused import * Fix various packages * Try to test Python packages directly in CI * Actually run the pipeline * Fix more packages * Fix mappings, fix packages * Fix dep version * Work around bug in concretizer * Various concretization fixes * Fix gitlab yaml, packages * Fix typo in gitlab yaml * Skip more packages that fail to concretize * Fix? jupyter ecosystem concretization issues * Solve Jupyter concretization issues * Prevent duplicate entries in PYTHONPATH * Skip fenics-dolfinx * Build fewer Python packages * Fix missing npm dep * Specify image * More package fixes * Add backends for every from-source package * Fix version arg * Remove GitLab CI stuff, add py-installer package * Remove test deps, re-add install_options * Function declaration syntax fix * More build fixes * Update spack create template * Update PythonPackage documentation * Fix documentation build * Fix unit tests * Remove pip flag added only in newer pip * flux: add explicit dependency on jsonschema * Update packages that have been added since this was branched off of develop * Move Python 2 deprecation to a separate PR * py-neurolab: add build dep on py-setuptools * Use wheels for pip/wheel * Allow use of pre-installed pip for external Python * pip -> python -m pip * Use python -m pip for all packages * Fix py-wrapt * Add both platlib and purelib to PYTHONPATH * py-pyyaml: setuptools is needed for all versions * py-pyyaml: link flags aren't needed * Appease spack audit packages * Some build backend is required for all versions, distutils -> setuptools * Correctly handle different setup.py filename * Use wheels for py-tomli to avoid circular dep on py-flit-core * Fix busco installation procedure * Clarify things in spack create template * Test other Python build backends * Undo changes to busco * Various fixes * Don't test other backends
2022-01-13Python: improve site_packages_dir handling (#28346)Adam J. Stewart3-33/+27
* Python: improve site_packages_dir handling * Replace all site_packages_dir with purelib/platlib
2022-01-12Do not initialize config on spack compiler list (#28042)Harmen Stoppels4-11/+37
When `spack compiler list` is run without being restricted to a particular scope, and no compilers are found, say that none are available, and hint that the use should run spack compiler find to auto detect compilers. * Improve docs * Check if stdin is a tty * add a test
2022-01-12bootstrap: fix bootstrapping GnuPG from different macOS versions (#28350)Massimiliano Culpo3-8/+27
2022-01-12unparser: handle package-level loops, if statements, and with blocksTodd Gamblin5-74/+1039
Many packages implement logic at the class level to handle complex dependencies and conflicts. Others have started using `with when("@1.0"):` blocks since we added that capability. The loops and other control logic can cause some pure directive logic not to be removed by our package hashing logic -- and in many cases that's a lot of code that will cause unnecessary rebuilds. This commit changes the unparser so that it will descend into these blocks. Specifically: 1. Descend into loops, if statements, and with blocks at the class level. 2. Don't look inside function definitions (in or outside a class). 3. Don't look at nested class definitions (they don't have directives) 4. Add logic to *remove* empty loops/with blocks/if statements if all directives in them were removed. This allows our package hash to ignore a lot of pure metadata that it was not ignoring before, and makes it less sensitive. In addition, we add `maintainers` and `tags` to the list of metadata attributes that Spack should remove from packages when constructing canonoical source for a package hash. - [x] Make unparser handle if/for/while/with at class level. - [x] Add tests for control logic removal. - [x] Add a test to ensure that all packages are not only unparseable, but also that their canonical source is still compilable. This is a test for our control logic removal. - [x] Add another unparse test package that has complex logic.
2022-01-12unparser: add unparser unit testsTodd Gamblin3-1/+528
These are the unit tests from astunparse, converted to pytest, with a few backports from upstream cpython. These should hopefully keep `unparser.py` well covered as we change it.
2022-01-12unparser: rename `t` to `node` to mirror upstreamTodd Gamblin1-359/+363
These refactors have happened in upstream `ast.unparse()`
2022-01-12unparser: rename `_Class()` methods to `visit_Class()` to mirror upstreamTodd Gamblin1-75/+75
These are refactors that have happened in upstream `ast.unparse()`
2022-01-12unparser: do a better job of roundtripping stringsTodd Gamblin1-19/+99
Handle complex f-strings. Backport of: https://github.com/python/cpython/commit/a993e901ebe60c38d46ecb31f771d0b4a206828c#
2022-01-12unparser: treat `print(a, b, c)` and `print((a, b, c))` the sameTodd Gamblin3-18/+453
We can't tell `print(a, b, c)` and `print((a, b, c))` apart -- both of these expressions generate different ASTs in Python 2 and Python 3. However, we can decide that we don't care. This commit treats both of them the same when `py_ver_consistent` is set with `unparse()`. This means that the package hash won't notice changes from printing a tuple to printing multiple values, but we don't care, because this is extremely unlikely to affect the build. More than likely this is just an error message for the user of the package. - [x] treat `print(a, b, c)` and `print((a, b, c))` the same in py2 and py3 - [x] add another package parsing test -- legion -- that exercises this feature
2022-01-12commands: add `spack pkg source` and `spack pkg hash`Todd Gamblin3-7/+128
To make it easier to see how package hashes change and how they are computed, add two commands: * `spack pkg source <spec>`: dumps source code for a package to the terminal * `spack pkg source --canonical <spec>`: dumps canonicalized source code for a package to the terminal. It strips comments, directives, and known-unused multimethods from the package. It is used to generate package hashes. * `spack pkg hash <spec>`: This gives the package hash for a particular spec. It is generated from the canonical source code for the spec. - [x] `add spack pkg source` and `spack pkg hash` - [x] add tests - [x] fix bug in multimethod resolution with boolean `@when` values Co-authored-by: Greg Becker <becker33@llnl.gov>
2022-01-12package_hash: switch to using canonical source instead of AST reprTodd Gamblin4-108/+120
We are planning to switch to using full hashes for Spack specs, which means that the package hash will be included in the deployment descriptor. This means we need a more robust package hash than simply dumping the `repr` of the AST. The AST repr that we previously used for package content is unreliable because it can vary between python versions (Python's AST actually changes fairly frequently). - [x] change `package_hash`, `package_ast`, and `canonical_source` to accept a string for alternate source instead of a filename. - [x] consolidate package hash tests in `test/util/package_hash.py`. - [x] remove old `package_content` method. - [x] make `package_hash` do what `canonical_source_hash` was doing before. - [x] modify `content_hash` in `package.py` to use the new `package_hash` function. Co-authored-by: Danny McClanahan <1305167+cosmicexplorer@users.noreply.github.com>
2022-01-12unparser: Don't omit parenthesis when unparsing a sliceTodd Gamblin1-14/+39
Backport of https://github.com/python/cpython/commit/c102a148256b00b7d48c51a1a97df19042e603de Includes support for Python 2.
2022-01-12unparser: Don't put unnecessary parentheses on class declarationsTodd Gamblin1-1/+1
Backport of * https://github.com/python/cpython/commit/25160cdc4775a1ddb4e37c8bf5a6e31ad9c146ed
2022-01-12package_hash: add test to ensure that every package in Spack can be unparsedTodd Gamblin2-6/+26
- [x] add option to canonical source to *not* filter multimethods - [x] add test to unparse every package in builtin
2022-01-12package_hash: add test to ensure consistency across Python versionsTodd Gamblin7-0/+2486
Our package hash is supposed to be consistent from python version to python version. Test this by adding some known unparse inputs and ensuring that they always have the same canonical hash. This test relies on the fact that we run Spack's unit tests across many python versions. We can't compute for several python versions within the same test run so we precompute the hashes and check them in CI.
2022-01-12package_hash: fix handling of multimethods and add testsTodd Gamblin2-35/+261
Package hashing was not properly handling multimethods. In particular, it was removing any functions that had decorators from the output, so we'd miss things like `@run_after("install")`, etc. There were also problems with handling multiple `@when`'s in a single file, and with handling `@when` functions that *had* to be evaluated dynamically. - [x] Rework static `@when` resolution for package hash - [x] Ensure that functions with decorators are not removed from output - [x] Add tests for many different @when scenarios (multiple @when's, combining with other decorators, default/no default, etc.) Co-authored-by: Danny McClanahan <1305167+cosmicexplorer@users.noreply.github.com>
2022-01-12package_hash: rework `RemoveDirectives` and add a testTodd Gamblin3-6/+41
Previously we used `directives.__all__` to get directive names, but it wasn't quite right -- it included `DirectiveMeta`, etc. It's not wrong, but it's also not the clearest way to do this. - [x] Refactor `@directive` to track names in `directive_names` global - [x] Rename `_directive_names` to `_directive_dict_names` in `DirectiveMeta` - [x] Add a test for `RemoveDirectives` Co-authored-by: Danny McClanahan <1305167+cosmicexplorer@users.noreply.github.com>
2022-01-12package_hash: remove all unassigned strings, not just docstringsTodd Gamblin2-9/+77
Some packages use top-level unassigned strings instead of comments, either just after a docstring on in the body somewhere else. Ignore those strings becasue they have no effect on package behavior. - [x] adjust RemoveDocstrings to remove all free-standing strings. - [x] move tests for util/package_hash.py to test/util/package_hash.py Co-authored-by: Danny McClanahan <1305167+cosmicexplorer@users.noreply.github.com>
2022-01-12unparser: handle unicode string literals consistently across Python versionsTodd Gamblin1-2/+16
Python 2 and 3 represent string literals differently in the AST. Python 2 requires '\x' literals, and Python 3 source is always unicode, and allows unicode to be written directly. These also unparse differently by default. - [x] modify unparser to write both out the way `repr` would in Python 2 when `py_ver_consistent` is provided.
2022-01-12unparser: implement operator precedence algorithm for unparserTodd Gamblin6-165/+310
Backport operator precedence algorithm from here: https://github.com/python/cpython/commit/397b96f6d7a89f778ebc0591e32216a8183fe667 This eliminates unnecessary parentheses from our unparsed output and makes Spack's unparser consistent with the one in upstream Python 3.9+, with one exception. Our parser normalizes argument order when `py_ver_consistent` is set, so that star arguments in function calls come last. We have to do this because Python 2's AST doesn't have information about their actual order. If we ever support only Python 3.9 and higher, we can easily switch over to `ast.unparse`, as the unparsing is consistent except for this detail (modulo future changes to `ast.unparse`)
2022-01-12unparser: refactor delimiting with context managers in ast.unparseTodd Gamblin1-190/+177
Backport of https://github.com/python/cpython/commit/4b3b1226e86df6cd45e921c8f2ad23c3639c43b2
2022-01-12unparser: add block() context manager for indentationTodd Gamblin1-63/+63
This is a backport of a refactor from cpython 3.9