summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2022-08-23meson: create unique names for build directory (#32062)Simon Pintarelli1-1/+9
Taken from CMakePackage
2022-08-23spack ci: add support for running stand-alone tests (#27877)Tamara Dahlgren22-445/+1669
This support requires adding the '--tests' option to 'spack ci rebuild'. Packages whose stand-alone tests are broken (in the CI environment) can be configured in gitlab-ci to be skipped by adding them to broken-tests-packages. Highlights include: - Restructured 'spack ci' help to provide better subcommand summaries; - Ensured only one InstallError (i.e., installer's) rather than allowing build_environment to have its own; and - Refactored CI and CDash reporting to keep CDash-related properties and behavior in a separate class. This allows stand-alone tests from `spack ci` to run when the `--tests` option is used. With `--tests`, stand-alone tests are run **after** a **successful** (re)build of the package. Test results are collected and report(able) using CDash. This PR adds the following features: - Adds `-t` and `--tests` to `spack ci rebuild` to run stand-alone tests; - Adds `--fail-fast` to stop stand-alone tests after the first failure; - Ensures a *single* `InstallError` across packages (i.e., removes second class from build environment); - Captures skipping tests for externals and uninstalled packages (for CDash reporting); - Copies test logs and outputs to the CI artifacts directory to facilitate debugging; - Parses stand-alone test results to report outputs from each `run_test` as separate test parts (CDash reporting); - Logs a test completion message to allow capture of timing of the last `run_test` part; - Adds the runner description to the CDash site to better distinguish entries in CDash tables; - Adds `gitlab-ci` `broken-tests-packages` to CI configuration to skip stand-alone testing for packages with known issues; - Changes `spack ci --help` so description of each subcommand is a single line; - Changes `spack ci <subcommand> --help` to provide the full description of each command (versus no description); and - Ensures `junit` test log file ends in an `.xml` extension (versus default where it does not). Tasks: - [x] Include the equivalent of the architecture information, or at least the host target, in the CDash output - [x] Upload stand-alone test results files as `test` artifacts - [x] Confirm tests are run in GitLab - [x] Ensure CDash results are uploaded as artifacts - [x] Resolve issues with CDash build-and test results appearing on same row of the table - [x] Add unit tests as needed - [x] Investigate why some (dependency) packages don't have test results (e.g., related from other pipelines) - [x] Ensure proper parsing and reporting of skipped tests (as `not run`) .. post- #28701 merge - [x] Restore the proper CDash URLand or mirror ONCE out-of-band testing completed
2022-08-20spack list: add --tag flag (#32016)sparkyniner2-0/+21
* modified list.py and added functionality for --tag * Removed long and very long, shifted rest of code above return statement * removed results variable * added import statement at top * added the line accidentally deleted * added line accidentally deleted * changed p.name to p, added line inside if statement * line order switched * [@spackbot] updating style on behalf of sparkyniner * ran update completion command * add tests * Update lib/spack/spack/test/cmd/list.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * [@spackbot] updating style on behalf of sparkyniner * changed argument to mock_packages and moved code under filter by tag * removed bad rebase code and added additional test * [@spackbot] updating style on behalf of sparkyniner * added line removed earlier * added line removed earlier * replaced function * added more recommended changes Co-authored-by: sairaj <sairaj@sairajs-MacBook-Pro.local> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2022-08-19docs: add a note about an issue being solved on `develop` (#32261)Harmen Stoppels1-1/+7
2022-08-19Add messages to assertions in asp.py (#32237)psakievich1-9/+40
Assertions without messages if/when hit create a blank error message for users. This PR adds error messages to all assertions in asp.py even if it seems unlikely they will ever be needed.
2022-08-18spack.util.package_hash: parametrize unit-tests (#32240)Massimiliano Culpo1-68/+29
* spack.util.package_hash: parametrize unit-tests * Fix comment
2022-08-18Remove unused argument in test fixture (#32236)Massimiliano Culpo1-1/+1
The argument is very likely a typo, and was meant to be given to the fixture decorator. Since the value being passed is the default, let's just remove it.
2022-08-18spec: fix typo in Spec._finalize_concretization (#32210)Massimiliano Culpo1-1/+1
2022-08-17bugfix: use cmake version from dependency (#31739)Tom Scogland2-7/+11
Ensure that build tools with module-level commands in spack use the version built as part of their build graph if one exists. This is now also required for mesa, scons, cmake and ctest, out of graph versions of these tools in path will not be found unless added as an external. This bug appeared because a new version of rocprim needs cmake 3.16, while I have 3.14 in my path I had added an external for cmake 3.20 to the dag, but 3.14 was still used to configure rocprim causing it to fail. As far as I can tell, all the build tools added in build_environment.py had this problem, despite the fact that they should have been resolving these tools by name with a path search and find the one in the dag that way. I'm still investigating why the path searching and Executable logic didn't do it, but this makes three of the build systems much more explicit, and leaves only gmake and ninja as dependencies from out in the system while ensuring the version in the dag is used if there is one. The additional sqlite version is to perturb the hash of python to work around a relocation bug which will be fixed in a subsequent PR.
2022-08-17Add c++14 support flag for xl compiler (#31826)Peter Scheibel1-0/+8
2022-08-16filesystem: in recursive mtime, check only files that exist (#32175)Wouter Deconinck1-1/+1
* filesystem: use lstat in recursive mtime When a `develop` path contains a dead symlink, the `os.stat` in the recursive `mtime` determination trips up over it. Closes #32165.
2022-08-16bugfix: silence `clingo` warning about `requirement_policy/3`Todd Gamblin1-0/+1
`requirement_policy/3` is generated and may not be in Spack's inputs to Clingo. Currently this is causing warnings like: ``` $ spack spec zlib /global/u2/t/tgamblin/src/spack/lib/spack/spack/solver/concretize.lp:510:3-43: info: atom does not occur in any rule head: requirement_policy(Package,X,"one_of") /global/u2/t/tgamblin/src/spack/lib/spack/spack/solver/concretize.lp:517:3-43: info: atom does not occur in any rule head: requirement_policy(Package,X,"one_of") /global/u2/t/tgamblin/src/spack/lib/spack/spack/solver/concretize.lp:523:3-43: info: atom does not occur in any rule head: requirement_policy(Package,X,"any_of") /global/u2/t/tgamblin/src/spack/lib/spack/spack/solver/concretize.lp:534:3-43: info: atom does not occur in any rule head: requirement_policy(Package,X,"any_of") Input spec -------------------------------- zlib Concretized -------------------------------- zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=cray-sles15-haswell ``` - [x] Silence warning with `#defined requirement_policy/3`
2022-08-16Configuration: allow users to enforce hard spec constraints (#27987)Peter Scheibel7-75/+545
Spack doesn't have an easy way to say something like "If I build package X, then I *need* version Y": * If you specify something on the command line, then you ensure that the constraints are applied, but the package is always built * Likewise if you `spack add X...`` to your environment, the constraints are guaranteed to hold, but the environment always builds the package * You can add preferences to packages.yaml, but these are not guaranteed to hold (Spack can choose other settings) This commit adds a 'require' subsection to packages.yaml: the specs added there are guaranteed to hold. The commit includes documentation for the feature. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-08-15CI/documentation: constrain pygments to avoid latest version (2.13.0) (#32163)Peter Scheibel1-0/+1
All PRs are failing the docs build on account of an error with pygments. These errors coincide with a new release of pygments (2.13.0) and restricting to < 2.13 allows the doc tests to pass, so this commit enforces that constraint for the docs build. Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2022-08-15Check that no reserved names are overridden by package recipes (#32092)Massimiliano Culpo1-0/+40
A few attribute in packages are meant to be reserved for Spack internal use. This audit checks packages to ensure none of these attributes are overridden. - [x] add additional audit check
2022-08-12rocm: add v5.2 for the entire stack (#31591)renjithravindrankannath1-8/+0
2022-08-11Fix performance regression with `spack mirror create --all` (#32005)Massimiliano Culpo6-169/+401
This PR fixes the performance regression reported in #31985 and a few other issues found while refactoring the spack mirror create command. Modifications: * (Primary) Do not require concretization for `spack mirror create --all` * Forbid using --versions-per-spec together with --all * Fixed a few issues when reading specs from input file (specs were not concretized, command would fail when trying to mirror dependencies) * Fix issue with default directory for spack mirror create not being canonicalized * Add more unit tests to poke spack mirror create * Skip externals also when mirroring environments * Changed slightly the wording for reporting (it was mentioning "Successfully created" even in presence of errors) * Fix issue with colify (was not called properly during error reporting)
2022-08-11CachedCMakePackage: Port over logic from CMake build system for ldlibs and ↵Chris White1-5/+26
ldflags (#31750) * port over logic from cmake build system for ldlibs and ldflags * follow new style standards * escape quotes * escape quotes * nevermind going back
2022-08-11Don't set `LD_LIBRARY_PATH` by default on Linux (#28354)Harmen Stoppels4-15/+26
`LD_LIBRARY_PATH` can break system executables (e.g., when an enviornment is loaded) and isn't necessary thanks to `RPATH`s. Packages that require `LD_LIBRARY_PATH` can set this in `setup_run_environment`. - [x] Prefix inspections no longer set `LD_LIBRARY_PATH` by default - [x] Document changes and workarounds for people who want `LD_LIBRARY_PATH`
2022-08-11Add missing info to diagnostic for circular provides (#32027)Chris Green1-1/+1
2022-08-11info: properly report default when for overridden variants (#32040)Tamara Dahlgren1-4/+2
2022-08-10Update tests so /tmp/store can be accessed (#32003)Richarda Butler2-13/+14
2022-08-10Patch /usr/bin/file to file when on nixos (#30718)Mikael Simberg1-0/+9
These changes make many packages build on nixos where nearly nothing comes from /bin or /usr/bin (the only things in "system locations" are /bin/sh and /usr/bin/env, all the rest is found through PATH). Many configuration scripts hardcode /usr/bin/file instead of using the one from PATH. This patches them to use file from PATH.
2022-08-10core: add a helpful long message for UnknownPackageError (#31996)Seth R. Johnson1-0/+2
``` Error: Package 'armpl' not found. ``` is pretty useless after an upgrade unless you're a spack pro. I've recently hit this on multiple machines. See https://github.com/spack/spack/issues/31453 , https://github.com/spack/spack/issues/31489 .
2022-08-10modules: add missing Python imports (#32012)Harmen Stoppels2-0/+4
2022-08-09Reduce installation noise: filter path padding and make some details debug ↵Tamara Dahlgren5-14/+61
level 2 (#31998)
2022-08-09PythonPackage: add --config-settings support (#31823)Adam J. Stewart2-4/+91
2022-08-09Add base class for directory visitor (#32008)Harmen Stoppels3-31/+103
2022-08-09Update release procedure, keep CHANGELOG up-to-date (#31969)Massimiliano Culpo1-5/+16
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-08-05spack mirror: skip non-concretizable specs (#31897)Massimiliano Culpo2-2/+38
fixes #31736 Catch errors when concretizing specs and report them as debug messages. The corresponding spec is skipped. Co-authored-by: Greg Becker <becker33@llnl.gov>
2022-08-05Refactor `cmd/install.py` for better readability (#31936)Massimiliano Culpo1-234/+252
* Extracted two functions in cmd/install.py * Extracted a function to perform installation from the active environment * Rename a few functions, remove args from their arguments * Rework conditional in install_from_active_environment to reduce nesting in the function * Extract functions to parsespecs from cli and files * Extract functions to getuser confirmation for overwrite * Extract functions to install specs inside and outside environments * Rename a couple of functions * Fix outdated comment * Add missing imports * Split conditional to dedent one level * Invert check and exit early to dedent one level when requiring user confirmation
2022-08-05Fix doc format for code blocks: GitRef versions (#31937)psakievich1-2/+2
Rendering was not pretty using `console` from #31914
2022-08-04Move type hint to satisfy Python 3.6, 3.7 interpreters (#31932)Chris Green1-2/+2
2022-08-04Add documentation for git refs as versions (#31914)psakievich3-1/+133
* document git commit versions Include documentation for manually specifying associated known version * document spack develop command Co-authorerd-by: psakievich <psakiev@sandia.gov> Co-authored-by: Gregory Becker <becker33@llnl.gov> Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-08-04Git Ref versions can be paired to defined versions in the spec (#30998)psakievich9-20/+145
The current use of git ref's as a version requires a search algorithm to pick the right matching version based on the tags in the git history of the package. This is less than ideal for the use case where users already know the specific version they want the git ref to be associated with. This PR makes a new version syntax [package]@[ref]=[version] to allow the users to specify the exact hash they wish to use.
2022-08-04Deprecate support for Python 3.5 (by removing it from CI) (#31908)Massimiliano Culpo2-2/+2
* Deprecate support for Python 3.5 (by removing it from CI) * Restore Python 3.9
2022-08-03asp: refactor low level API to permit the injection of configurationMassimiliano Culpo4-77/+78
This allows writing extension commands that can benchmark different configurations in clingo, or try different configurations for a single test.
2022-08-03Match documentation of spack create with actual behavior. (#31892)dunatotatos1-1/+1
2022-08-02spack ci rebuild: Don't install in a login shell (#31771)Jonathon Anderson1-3/+2
On some systems the shell in login mode wipes important parts of the environment, such as PATH. This causes the build to fail since it can't find `spack`. For better robustness, don't use a login shell. In a full CI job the final spack install is run in an environment formed by scripts running in this order: export AWS_SECRET=... # 1. Load environment from GitLab project variables source spack/share/spack/setup-env.sh # 2. Load Spack into the environment (PATH) spack env activate -V concrete_env # 3. Activate the concrete environment source /etc/profile # 4. Bash login shell (from -l) spack install ... Whereas when a user launches their own container with (docker|podman) run -it, they end up running spack install in an environment formed in this order: source /etc/bash.bashrc # (not 4). Bash interactive shell (default with TTY) export AWS_SECRET=... #~1. Manually load environment from GitLab project variables source spack/share/spack/setup-env.sh # 2. Load Spack into the environment (PATH) spack env activate -V concrete_env # 3. Activate the concrete environment spack install ... The big problem being that (4) has a completely different position and content (on Leap 15 and possibly other containers). So in context, this PR removes (4) from the CI job case, leaving us with the simpler: export AWS_SECRET=... # 1. Load environment from GitLab project variables source spack/share/spack/setup-env.sh # 2. Load Spack into the environment (PATH) spack env activate -V concrete_env # 3. Activate the concrete environment spack install ...
2022-08-02Optimize reuse from buildcaches (#30806)Massimiliano Culpo2-11/+13
* database: don't sort on return from query_local * ASP-based solver: don't build the hash-lookup dictionary twice Building this dictionary twice and traversing all the specs might be time-consuming for large buildcaches.
2022-08-02Style fixesMassimiliano Culpo3-10/+11
2022-08-02Remove line that sets non-existing attribute?Massimiliano Culpo1-1/+0
2022-08-02Use __slots__ for fast attribute accessMassimiliano Culpo2-0/+12
2022-08-02Use __slots__ for fast attribute access during parsingMassimiliano Culpo2-0/+8
2022-08-02Document that reuse is higher priority than preferences in packages.yaml ↵Harmen Stoppels2-0/+35
(#31864) Resurrect Known issues, since users ask frequently about that.
2022-08-02docs: Fix outdated pre-0.18 text on build dependencies (#31860)Todd Gamblin1-5/+13
2022-08-02Add meson Python build backend (#31809)Adam J. Stewart1-36/+51
2022-08-02spack checksum: black compliance (#31851)Adam J. Stewart1-1/+1
2022-08-01Add c++14 support flag for xl compiler (#31826)Peter Scheibel1-0/+8
2022-08-01Add support for macOS Ventura (#31812)Adam J. Stewart1-0/+1