summaryrefslogtreecommitdiff
path: root/var/spack/repos/builtin.mock
AgeCommit message (Collapse)AuthorFilesLines
2024-01-02Update copyright year to 2024 (#41919)Todd Gamblin302-302/+302
It was time to run `spack license update-copyright-year` again.
2023-11-07Propagate variant across nodes that don't have that variant (#38512)Richarda Butler5-0/+104
Before this PR, variant were not propagated to leaf nodes that could accept the propagated value, if some intermediate node couldn't accept it. This PR fixes that issue by marking nodes as "candidate" for propagation and by setting the variant only if it can be accepted by the node. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-11-06Don't let runtime env variables of compiler like deps leak into the build ↵Harmen Stoppels2-0/+34
environment (#40916) * Test that setup_run_environment changes to CC/CXX/FC/F77 are dropped in build env * compilers set in run env shouldn't impact build Adds `drop` to EnvironmentModifications courtesy of @haampie, and uses it to clear modifications of CC, CXX, F77 and FC made by `setup_{,dependent_}run_environment` routines when producing an environment in BUILD context. * comment / style * comment --------- Co-authored-by: Tom Scogland <scogland1@llnl.gov>
2023-11-03Bugfix: propagation of multivalued variants (#39833)Richarda Butler2-1/+5
Don't encourage use of default value if propagating a multivalued variant.
2023-11-01Cherry-picking virtual dependencies (#35322)Massimiliano Culpo5-5/+43
This PR makes it possible to select only a subset of virtual dependencies from a spec that _may_ provide more. To select providers, a syntax to specify edge attributes is introduced: ``` hdf5 ^[virtuals=mpi] mpich ``` With that syntax we can concretize specs like: ```console $ spack spec strumpack ^[virtuals=mpi] intel-parallel-studio+mkl ^[virtuals=lapack] openblas ``` On `develop` this would currently fail with: ```console $ spack spec strumpack ^intel-parallel-studio+mkl ^openblas ==> Error: Spec cannot include multiple providers for virtual 'blas' Requested 'intel-parallel-studio' and 'openblas' ``` In package recipes, virtual specs that are declared in the same `provides` directive need to be provided _together_. This means that e.g. `openblas`, which has: ```python provides("blas", "lapack") ``` needs to provide both `lapack` and `blas` when requested to provide at least one of them. ## Additional notes This capability is needed to model compilers. Assuming that languages are treated like virtual dependencies, we might want e.g. to use LLVM to compile C/C++ and Gnu GCC to compile Fortran. This can be accomplished by the following[^1]: ``` hdf5 ^[virtuals=c,cxx] llvm ^[virtuals=fortran] gcc ``` [^1]: We plan to add some syntactic sugar around this syntax, and reuse the `%` sigil to avoid having a lot of boilerplate around compilers. Modifications: - [x] Add syntax to interact with edge attributes from spec literals - [x] Add concretization logic to be able to cherry-pick virtual dependencies - [x] Extend semantic of the `provides` directive to express when virtuals need to be provided together - [x] Add unit-tests and documentation
2023-10-31Fix interaction of spec literals that propagate variants with unify:false ↵Massimiliano Culpo2-0/+38
(#40789) * Add tests to ensure variant propagation syntax can round-trip to/from string * Add a regression test for the bug in 35298 * Reconstruct the spec constraints in the worker process Specs do not preserve any information on propagation of variants when round-tripping to/from JSON (which we use to pickle), but preserve it when round-tripping to/from strings. Therefore, we pass a spec literal to the worker and reconstruct the Spec objects there.
2023-10-23audit: add check for GitLab patches (#40656)Michael Kuhn2-0/+40
GitLab's .patch URLs only provide abbreviated hashes, while .diff URLs provide full hashes. There does not seem to be a parameter to force .patch URLs to also return full hashes, so we should make sure to use the .diff ones.
2023-10-18AutotoolsPackage / MakefilePackage: add gmake build dependency (#40380)Harmen Stoppels3-0/+36
2023-10-18Add license directive (#39346)Aiden Grossman1-0/+18
This patch adds in a license directive to get the ball rolling on adding in license information about packages to spack. I'm primarily interested in just adding license into spack, but this would also help with other efforts that people are interested in such as adding license information to the ASP solve for concretization to make sure licenses are compatible. Usage: Specifying the specific license that a package is released under in a project's `package.py` is good practice. To specify a license, find the SPDX identifier for a project and then add it using the license directive: ```python license("<SPDX Identifier HERE>") ``` For example, for Apache 2.0, you might write: ```python license("Apache-2.0") ``` Note that specifying a license without a when clause makes it apply to all versions and variants of the package, which might not actually be the case. For example, a project might have switched licenses at some point or have certain build configurations that include files that are licensed differently. To account for this, you can specify when licenses should be applied. For example, to specify that a specific license identifier should only apply to versionup to and including 1.5, you could write the following directive: ```python license("MIT", when="@:1.5") ```
2023-08-28Patch VTK to enable python 3.8 in VTK 8.2 (#38735)kwryankrattiger1-0/+1
* VTK: Add patch for python 3.8 support * CI: Re-enable VisIt in CI * Configure spec matrix for stack with VisIt * Add pugixml dep for 8.2.0 * Make VTK and ParaView consistent on proj dep * OpenMPI 3: provides MP support by default * Add details on proj dep in ParaView * Add python 3.8 to test mock repo * Patches to get VisIt VTK interface * CI: Disable VisIt with GUI in DAV
2023-08-15Rework conflicts so that "vendors" is not needed anymoreMassimiliano Culpo1-2/+3
2023-08-15Add vendors directiveMassimiliano Culpo1-0/+18
For the time being this directive prevents the vendored package to be in the same DAG as the one vendoring it.
2023-08-14Windows: executable/path handling (#37762)markus-ferrell1-1/+1
Windows executable paths can have spaces in them, which was leading to errors when constructing Executable objects: the parser was intended to handle cases where users could provide an executable along with one or more space-delimited arguments. * Executable now assumes that it is constructed with a string argument that represents the path to the executable, but no additional arguments. * Invocations of Executable.__init__ that depended on this have been updated (this includes the core, tests, and one instance of builtin repository package). * The error handling for failed invocations of Executable.__call__ now includes a check for whether the executable name/path contains a space, to help users debug cases where they (now incorrectly) concatenate the path and the arguments.
2023-08-14Windows: enable "spack install" tests (#34696)markus-ferrell3-31/+73
* The module-level skip for tests in `cmd.install` on Windows is removed. A few classes of errors still persist: * Cdash tests are not working on Windows * Tests for failed installs are also not working (this will require investigating bugs in output redirection) * Environments are not yet supported on Windows overall though, this enables testing of most basic uses of "spack install" * Git repositories cached for version lookups were using a layout that mimicked the URL as much as possible. This was useful for listing the cache directory and understanding what was present at a glance, but the paths were overly long on Windows. On all systems, the layout is now a single directory based on a hash of the Git URL and is shortened (which ensures a consistent and acceptable length, and also avoids special characters). * In particular, this removes util.url.parse_git_url and its associated test, which were used exclusively for generating the git cache layout * Bootstrapping is now enabled for unit tests on Windows
2023-07-19modules: use curly braces to enclose value in Tcl modulefile (#38375)Xavier Delaruelle1-0/+19
Use curly braces instead of quotes to enclose value or text in Tcl modulefile. Within curly braces Tcl special characters like [, ] or $ are treated verbatim whereas they are evaluated within quotes. Curly braces is Tcl recommended way to enclose verbatim content [1]. Note: if curly braces charaters are used within content, they must be balanced. This point has been checked against current repository and no unbalanced curly braces has been spotted. Fixes #24243 [1] https://wiki.tcl-lang.org/page/Tcl+Minimal+Escaping+Style
2023-07-19Remove `LazyReference` from code (#38944)Massimiliano Culpo1-1/+1
A LazyReference object is a reference to an attribute of a lazily evaluated singleton. Its only purpose is to let developers use shorter names to refer to such attribute. This class does more harm than good, as it obfuscates the fact that we are using the attribute of a global object. Also, it can easily go out of sync with the singleton it refers to if, for instance, the singleton is updated but the references are not. This commit removes the LazyReference class entirely, and access the attributes explicitly passing through the global value to which they are attached.
2023-07-12vtk-m: correct cuda_arch variant behavior (#38697)Vicente Bolea1-0/+37
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
2023-07-05Drop Python 2 super syntax (#38718)Adam J. Stewart2-2/+2
2023-06-26Add raw attribute to env.set command (#38465)Xavier Delaruelle1-0/+16
Update `env.set` command and underlying `SetEnv` object to add the `raw` boolean attribute. `raw` is optional and set to False by default. When set to True, value format is skipped for object when generating environment modifications. With this change it is now possible to define environment variable whose value contains variable reference syntax (like `{foo}` or `{}`) that should be set as-is. Fixes #29578
2023-06-13modules: append trailing delimiter to MANPATH when set (#36678)Xavier Delaruelle3-0/+49
Update modulefile templates to append a trailing delimiter to MANPATH environment variable, if the modulefile sets it. With a trailing delimiter at ends of MANPATH's value, man will search the system man pages after searching the specific paths set. Using append-path/append_path to add this element, the module tool ensures it is appended only once. When modulefile is unloaded, the number of append attempt is decreased, thus the trailing delimiter is removed only if this number equals 0. Disclaimer: no path element should be appended to MANPATH by generated modulefiles. It should always be prepended to ensure this variable's value ends with the trailing delimiter. Fixes #11355.
2023-05-17spack test: fix stand-alone test suite status reporting (#37602)Tamara Dahlgren1-3/+11
* Fix reporting of packageless specs as having no tests * Add test_test_output_multiple_specs with update to simple-standalone-test (and tests) * Refactored test status summary; added more tests or checks
2023-05-16Requirements and preferences should not define (non-git) versions (#37687)Peter Scheibel1-0/+1
Ensure that requirements `packages:*:require:@x` and preferences `packages:*:version:[x]` fail concretization when no version defined in the package satisfies `x`. This always holds except for git versions -- they are defined on the fly.
2023-05-10Stand-alone testing: make recipe support and processing spack-/pytest-like ↵Tamara Dahlgren8-15/+71
(#34236) This is a refactor of Spack's stand-alone test process to be more spack- and pytest-like. It is more spack-like in that test parts are no longer "hidden" in a package's run_test() method and pytest-like in that any package method whose name starts test_ (i.e., a "test" method) is a test part. We also support the ability to embed test parts in a test method when that makes sense. Test methods are now implicit test parts. The docstring is the purpose for the test part. The name of the method is the name of the test part. The working directory is the active spec's test stage directory. You can embed test parts using the test_part context manager. Functionality added by this commit: * Adds support for multiple test_* stand-alone package test methods, each of which is an implicit test_part for execution and reporting purposes; * Deprecates package use of run_test(); * Exposes some functionality from run_test() as optional helper methods; * Adds a SkipTest exception that can be used to flag stand-alone tests as being skipped; * Updates the packaging guide section on stand-alone tests to provide more examples; * Restores the ability to run tests "inherited" from provided virtual packages; * Prints the test log path (like we currently do for build log paths); * Times and reports the post-install process (since it can include post-install tests); * Corrects context-related error message to distinguish test recipes from build recipes.
2023-05-08Add a "requires" directive, extend functionality of package requirements ↵Massimiliano Culpo2-0/+36
(#36286) Add a "require" directive to packages, which functions exactly like requirements specified in packages.yaml (uses the same fact-generation logic); update both to allow making the requirement conditional. * Packages may now use "require" to add constraints. This can be useful for something like "require(%gcc)" (where before we had to add a conflict for every compiler except gcc). * Requirements (in packages.yaml or in a "require" directive) can be conditional on a spec, e.g. "require(%gcc, when=@1.0.0)" (version 1.0.0 can only build with gcc). * Requirements may include a message which clarifies why they are needed. The concretizer assigns a high priority to errors which generate these messages (in particular over errors for unsatisfied requirements that do not produce messages, but also over a number of more-generic errors).
2023-05-05Improve version, version range, and version list syntax and behavior (#36273)Harmen Stoppels12-25/+25
## Version types, parsing and printing - The version classes have changed: `VersionBase` is removed, there is now a `ConcreteVersion` base class. `StandardVersion` and `GitVersion` both inherit from this. - The public api (`Version`, `VersionRange`, `ver`) has changed a bit: 1. `Version` produces either `StandardVersion` or `GitVersion` instances. 2. `VersionRange` produces a `ClosedOpenRange`, but this shouldn't affect the user. 3. `ver` produces any of `VersionList`, `ClosedOpenRange`, `StandardVersion` or `GitVersion`. - No unexpected type promotion, so that the following is no longer an identity: `Version(x) != VersionRange(x, x)`. - `VersionList.concrete` now returns a version if it contains only a single element subtyping `ConcreteVersion` (i.e. `StandardVersion(...)` or `GitVersion(...)`) - In version lists, the parser turns `@x` into `VersionRange(x, x)` instead of `Version(x)`. - The above also means that `ver("x")` produces a range, whereas `ver("=x")` produces a `StandardVersion`. The `=` is part of _VersionList_ syntax. - `VersionList.__str__` now outputs `=x.y.z` for specific version entries, and `x.y.z` as a short-hand for ranges `x.y.z:x.y.z`. - `Spec.format` no longer aliases `{version}` to `{versions}`, but pulls the concrete version out of the list and prints that -- except when the list is is not concrete, then is falls back to `{versions}` to avoid a pedantic error. For projections of concrete specs, `{version}` should be used to render `1.2.3` instead of `=1.2.3` (which you would get with `{versions}`). The default `Spec` format string used in `Spec.__str__` now uses `{versions}` so that `str(Spec(string)) == string` holds. ## Changes to `GitVersion` - `GitVersion` is a small wrapper around `StandardVersion` which enriches it with a git ref. It no longer inherits from it. - `GitVersion` _always_ needs to be able to look up an associated Spack version if it was not assigned (yet). It throws a `VersionLookupError` whenever `ref_version` is accessed but it has no means to look up the ref; in the past Spack would not error and use the commit sha as a literal version, which was incorrect. - `GitVersion` is never equal to `StandardVersion`, nor is satisfied by it. This is such that we don't lose transitivity. This fixes the following bug on `develop` where `git_version_a == standard_version == git_version_b` does not imply `git_version_a == git_version_b`. It also ensures equality always implies equal hash, which is also currently broken on develop; inclusion tests of a set of versions + git versions would behave differently from inclusion tests of a list of the same objects. - The above means `ver("ref=1.2.3) != ver("=1.2.3")` could break packages that branch on specific versions, but that was brittle already, since the same happens with externals: `pkg@1.2.3-external` suffixes wouldn't be exactly equal either. Instead, those checks should be `x.satisfies("@1.2.3")` which works both for git versions and custom version suffixes. - `GitVersion` from commit will now print as `<hash>=<version>` once the git ref is resolved to a spack version. This is for reliability -- version is frozen when added to the database and queried later. It also improves performance since there is no need to clone all repos of all git versions after `spack clean -m` is run and something queries the database, triggering version comparison, such as potentially reuse concretization. - The "empty VerstionStrComponent trick" for `GitVerison` is dropped since it wasn't representable as a version string (by design). Instead, it's replaced by `git`, so you get `1.2.3.git.4` (which reads 4 commits after a tag 1.2.3). This means that there's an edge case for version schemes `1.1.1`, `1.1.1a`, since the generated git version `1.1.1.git.1` (1 commit after `1.1.1`) compares larger than `1.1.1a`, since `a < git` are compared as strings. This is currently a wont-fix edge case, but if really required, could be fixed by special casing the `git` string. - Saved, concrete specs (database, lock file, ...) that only had a git sha as their version, but have no means to look the effective Spack version anymore, will now see their version mapped to `hash=develop`. Previously these specs would always have their sha literally interpreted as a version string (even when it _could_ be looked up). This only applies to databases, lock files and spec.json files created before Spack 0.20; after this PR, we always have a Spack version associated to the relevant GitVersion). - Fixes a bug where previously `to_dict` / `from_dict` (de)serialization would not reattach the repo to the GitVersion, causing the git hash to be used as a literal (bogus) version instead of the resolved version. This was in particularly breaking version comparison in the build process on macOS/Windows. ## Installing or matching specific versions - In the past, `spack install pkg@3.2` would install `pkg@=3.2` if it was a known specific version defined in the package, even when newer patch releases `3.2.1`, `3.2.2`, `...` were available. This behavior was only there because there was no syntax to distinguish between `3.2` and `3.2.1`. Since there is syntax for this now through `pkg@=3.2`, the old exact matching behavior is removed. This means that `spack install pkg@3.2` constrains the `pkg` version to the range `3.2`, and `spack install pkg@=3.2` constrains it to the specific version `3.2`. - Also in directives such as `depends_on("pkg@2.3")` and their when conditions `conflicts("...", when="@2.3")` ranges are ranges, and specific version matches require `@=2.3.`. - No matching version: in the case `pkg@3.2` matches nothing, concretization errors. However, if you run `spack install pkg@=3.2` and this version doesn't exist, Spack will define it; this allows you to install non-registered versions. - For consistency, you can now do `%gcc@10` and let it match a configured `10.x.y` compiler. It errors when there is no matching compiler. In the past it was interpreted like a specific `gcc@=10` version, which would get bootstrapped. - When compiler _bootstrapping_ is enabled, `%gcc@=10.2.0` can be used to bootstrap a specific compiler version. ## Other changes - Externals, compilers, and develop spec definitions are backwards compatible. They are typically defined as `pkg@3.2.1` even though they should be saying `pkg@=3.2.1`. Spack now transforms `pkg@3` into `pkg@=3` in those cases. - Finally, fix strictness of `version(...)` directive/declaration. It just does a simple type check, and now requires strings/integers. Floats are not allowed because they are ambiguous `str(3.10) == "3.1"`.
2023-05-02GitHub Actions: do not install six in CI (#37361)Massimiliano Culpo2-4/+2
* GitHub Actions: do not install six in CI * Remove workflow code that was commented out * Remove any use of "six" from packages
2023-04-19make version(...) kwargs explicit (#36998)Harmen Stoppels197-334/+338
- [x] Replace `version(ver, checksum=None, **kwargs)` signature with `version(ver, checksum=None, *, sha256=..., ...)` explicitly listing all arguments. - [x] Fix various issues in packages: - `tags` instead of `tag` - `default` instead of `preferred` - `sha26` instead of `sha256` - etc Also, use `sha256=...` consistently. Note: setting `sha256` currently doesn't validate the checksum length, so you could do `sha256="a"*32` and it would get checked as `md5`... but that's something for another PR.
2023-03-24modules: add support for append_flags/remove_flags (#36402)Xavier Delaruelle1-0/+3
Adapt tcl and lmod modulefile templates to generate append-path or remove-path commands in modulefile when respectively append_flags or remove_flags commands are defined in package for run environment. Fixes #10299.
2023-03-19Fix case spelling for Lmod and Tcl (#36215)Xavier Delaruelle1-1/+1
2023-03-18cmake build system: make "generator" a variant (#35552)Massimiliano Culpo1-0/+15
2023-03-18Fix wrong computation of concrete specs due to a bug in intersects (#36194)Massimiliano Culpo1-0/+2
fixes #36190
2023-03-08Split `satisfies(..., strict=True/False)` into two functions (#35681)Massimiliano Culpo1-0/+18
This commit formalizes `satisfies(lhs, rhs, strict=True/False)` and splits it into two functions: `satisfies(lhs, rhs)` and `intersects(lhs, rhs)`. - `satisfies(lhs, rhs)` means: all concrete specs matching the left hand side also match the right hand side - `intersects(lhs, rhs)` means: there exist concrete specs matching both lhs and rhs. `intersects` now has the property that it's commutative, which previously was not guaranteed. For abstract specs, `intersects(lhs, rhs)` implies that `constrain(lhs, rhs)` works. What's *not* done in this PR is ensuring that `intersects(concrete, abstract)` returns false when the abstract spec has additional properties not present in the concrete spec, but `constrain(concrete, abstract)` will raise an error. To accomplish this, some semantics have changed, as well as bugfixes to ArchSpec: - GitVersion is now interpreted as a more constrained version - Compiler flags are interpreted as strings since their order is important - Abstract specs respect variant type (bool / multivalued)
2023-03-05windows: use `sys.platform == "win32"` instead of `is_windows` (#35640)Todd Gamblin1-3/+1
`mypy` only understands `sys.platform == "win32"`, not indirect assignments of that value to things like `is_windows`. If we don't use the accepted platform checks, `mypy` registers many Windows-only symbols as not present on Linux, when it should skip the checks for platform-specific code.
2023-02-16Style: black 23, skip magic trailing comma (#35351)Adam J. Stewart5-14/+4
* Style: black 23, skip magic trailing commas * isort should use same line length as black * Fix unused import * Update version of black used in CI * Update new packages * Update new packages
2023-02-16spack uninstall: follow run/link edges on --dependents (#34058)Harmen Stoppels4-0/+67
`spack gc` removes build deps of explicitly installed specs, but somehow if you take one of the specs that `spack gc` would remove, and feed it to `spack uninstall /<hash>` by hash, it complains about all the dependents that still rely on it. This resolves the inconsistency by only following run/link type deps in spack uninstall. That way you can finally do `spack uninstall cmake` without having to remove all packages built with cmake.
2023-02-01Use the `maintainers` directive in all packages (#35201)Massimiliano Culpo2-2/+2
2023-01-27Add a `maintainers` directive (#35083)Massimiliano Culpo3-3/+19
fixes #34879 This commit adds a new maintainer directive, which by default extend the list of maintainers for a given package. The directive is backward compatible with the current practice of having a "maintainers" list declared at the class level.
2023-01-20Windows/testing: enable tests for "spack find" (#33588)markus-ferrell2-6/+4
2023-01-20Identify Windows support with tags (#35027)John W. Parent1-0/+1
All packages with explicit Windows support can be found with `spack list --tags=windows`. This also removes the documentation which explicitly lists supported packages on Windows (which is currently out of date and is now unnecessary with the added tags). Note that if a package does not appear in this list, it *may* still build on Windows, but it likely means that no explicit attempt has been made to support it.
2023-01-18license year bump (#34921)Harmen Stoppels268-268/+268
* license bump year * fix black issues of modified files * mypy * fix 2021 -> 2023
2022-12-09Windows: reenable unit tests (#33385)John W. Parent1-3/+11
Unit tests on Windows are supposed to pass for any PR to pass CI. However, the return code for the unit test command was not being checked, which meant this check was always passing (effectively disabled). This PR * Properly checks the result of the unit tests and fails if the unit tests fail * Fixes (or disables on Windows) a number of tests which have "drifted" out of support on Windows since this check was effectively disabled
2022-12-01conditional variant values: allow boolean (#33939)Greg Becker1-0/+7
2022-11-15Fix incorrect timer (#33900)Harmen Stoppels1-5/+0
Revamp the timer so we always have a designated begin and end. Fix a bug where the phase timer was stopped before the phase started, resulting in incorrect timing reports in timers.json.
2022-11-14Speed up traverse unit tests (#33840)Harmen Stoppels4-129/+0
2022-11-07intel oneapi classic bootstrapping (#31285)Greg Becker1-0/+28
The `intel` compiler at versions > 20 is provided by the `intel-oneapi-compilers-classic` package (a thin wrapper around the `intel-oneapi-compilers` package), and the `oneapi` compiler is provided by the `intel-oneapi-compilers` package. Prior to this work, neither of these compilers could be bootstrapped by Spack as part of an install with `install_missing_compilers: True`. Changes made to make these two packages bootstrappable: 1. The `intel-oneapi-compilers-classic` package includes a bin directory and symlinks to the compiler executables, not just logical pointers in Spack. 2. Spack can look for bootstrapped compilers in directories other than `$prefix/bin`, defined on a per-package basis 3. `intel-oneapi-compilers` specifies a non-default search directory for the compiler executables. 4. The `spack.compilers` module now can make more advanced associations between packages and compilers, not just simple name translations 5. Spack support for lmod hierarchies accounts for differences between package names and the associated compiler names for `intel-oneapi-compilers/oneapi`, `intel-oneapi-compilers-classic/intel@20:`, `llvm+clang/clang`, and `llvm-amdgpu/rocmcc`. - [x] full end-to-end testing - [x] add unit tests
2022-11-08Fix missing "*.spack*" files in views (#30980)Jordan Galby1-0/+46
All files/dirs containing ".spack" anywhere their name were ignored when generating a spack view. For example, this happened with the 'r' package.
2022-11-05Fix relocation to avoid overwriting merged constant strings (#32253)Tom Scogland2-2/+12
Compilers and linker optimize string constants for space by aliasing them when one is a suffix of another. For gcc / binutils this happens already at -O1, due to -fmerge-constants. This means that we have to take care during relocation to always preserve a certain length of the suffix of those prefixes that are C-strings. In this commit we pick length 7 as a safe suffix length, assuming the suffix is typically the 7 characters from the hash (i.e. random), so it's unlikely to alias with any string constant used in the sources. In general we now pad shortened strings from the left with leading dir seperators, but in the case of C-strings that are much shorter and don't share a common suffix (due to projections), we do allow shrinking the C-string, appending a null, and retaining the old part of the prefix. Also when rewiring, we ensure that the new hash preserves the last 7 bytes of the old hash. Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-11-02flags/variants: Add ++/~~/== syntax for propagation to dependenciesKayla Butler2-0/+9
Currently, compiler flags and variants are inconsistent: compiler flags set for a package are inherited by its dependencies, while variants are not. We should have these be consistent by allowing for inheritance to be enabled or disabled for both variants and compiler flags. - [x] Make new (spec language) operators - [x] Apply operators to variants and compiler flags - [x] Conflicts currently result in an unsatisfiable spec (i.e., you can't propagate two conflicting values) What I propose is using two of the currently used sigils to symbolized that the variant or compiler flag will be inherited: Example syntax: - `package ++variant` enabled variant that will be propagated to dependencies - `package +variant` enabled variant that will NOT be propagated to dependencies - `package ~~variant` disabled variant that will be propagated to dependencies - `package ~variant` disabled variant that will NOT be propagated to dependencies - `package cflags==True` `cflags` will be propagated to dependencies - `package cflags=True` `cflags` will NOT be propagated to dependencies Syntax for string-valued variants is similar to compiler flags.
2022-11-01Consolidate DAG traversal in traverse.py, support DFS/BFS (#33406)Harmen Stoppels4-0/+129
This PR introduces breadth-first traversal, and moves depth-first traversal logic out of Spec's member functions, into `traverse.py`. It introduces a high-level API with three main methods: ```python spack.traverse.traverse_edges(specs, kwargs...) spack.traverse.traverse_nodes(specs, kwags...) spack.traverse.traverse_tree(specs, kwargs...) ``` with the usual `root`, `order`, `cover`, `direction`, `deptype`, `depth`, `key`, `visited` kwargs for the first two. What's new is that `order="breadth"` is added for breadth-first traversal. The lower level API is not exported, but is certainly useful for advanced use cases. The lower level API includes visitor classes for direction reversal and edge pruning, which can be used to create more advanced traversal methods, especially useful when the `deptype` is not constant but depends on the node or depth. --- There's a couple nice use-cases for breadth-first traversal: - Sometimes roots have to be handled differently (e.g. follow build edges of roots but not of deps). BFS ensures that root nodes are always discovered at depth 0, instead of at any depth > 1 as a dep of another root. - When printing a tree, it would be nice to reduce indent levels so it fits in the terminal, and ensure that e.g. `zlib` is not printed at indent level 10 as a dependency of a build dep of a build dep -- rather if it's a direct dep of my package, I wanna see it at depth 1. This basically requires one breadth-first traversal to construct a tree, which can then be printed with depth-first traversal. - In environments in general, it's sometimes inconvenient to have a double loop: first over the roots then over each root's deps, and maintain your own `visited` set outside. With BFS, you can simply init the queue with the environment root specs and it Just Works. [Example here](https://github.com/spack/spack/blob/3ec73046995d9504d6e135f564f1370cfe31ba34/lib/spack/spack/environment/environment.py#L1815-L1816)
2022-11-01Unit tests: make unit tests work for aarch64 machines (#33625)Greg Becker3-0/+7
Currently, many tests hardcode to older versions of gcc for comparisons of concretization among compiler versions. Those versions are too old to concretize for `aarch64`-family targets, which leads to failing tests on `aarch64`. This PR fixes those tests by updating the compiler versions used for testing. Currently, many tests hardcode the expected architecture result in concretization to the `x86_64` family of architectures. This PR generalizes the tests that can be generalized, to cover multiple architecture families. For those that test specific relationships among `x86_64`-family targets, it ensures that concretization uses the `x86_64`-family targets in those cases. Currently, many tests rely on the fact that `AutotoolsPackage` imposes no dependencies on the inheriting package. That is not true on `aarch64`-family architectures. This PR ensures that the fact `AutotoolsPackage` on `aarch64` pulls in a dependency on `gnuconfig` is ignored when testing for the appropriate relationships among dependencies Additionally, 5 tests currently prompt the user for input when `gpg` is available in the user's path. This PR fixes that issue. And 7 tests fail currently when the user has a yubikey available. This PR fixes the incorrect gpg argument causing those issues.