summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2023-07-04Remove from __future__ imports (#38703)Adam J. Stewart61-125/+1
2023-07-02SIPPackage: documentation fix (#38672)Adam J. Stewart1-1/+1
2023-06-30gitlab ci: reorganize when we check for specs on mirrors (#38626)Scott Wittenburg1-86/+85
Move the logic checking which mirrors have the specs we need closer to where that information is needed. Also update the staging summary to contain a brief description of why we scheduled or pruned each job. If a spec was found on any mirrors, regardless of whether we scheduled a job for it, print those mirrors.
2023-06-30tests/pythons: convert to new stand-alone test process (#38340)Tamara Dahlgren1-6/+9
2023-06-27Windows: Add PowerShell env support (#37951)John W. Parent7-13/+59
PowerShell requires explicit shell and env support in Spack. This is due to the distinct differences in shell interactions between cmd and pwsh. Add a doskey in pwsh piping 'spack' commands to a powershell script similar to the sh function 'spack'. Add support for PowerShell-specific shell interactions from Spack (set/unset shell variables).
2023-06-27ci: remove compiler bootstrapping code (#38543)Scott Wittenburg5-742/+193
2023-06-27Adjust cc90 to require at least CUDA 12 (#38572)Wileam Y. Phan1-3/+2
2023-06-27Revert "Updates to symlinking for Windows (#34701)" (#38578)Harmen Stoppels13-817/+191
This reverts commit 66f75407d136c0c06ed01774d5815728afba35ca.
2023-06-26CachedCMakePackage: set build type (#38502)Brian Van Essen1-0/+1
Fixed the cached CMake package so that the build_type field is saved in the cached configuration file.
2023-06-26Updates to symlinking for Windows (#34701)David Alexander13-191/+817
* Support hardlinks/junctions on Windows systems without developer mode enabled * Generally, use of llnl.util.symlink.symlink is preferred over os.symlink since it handles this automatically * Generally an error is now reported if a user attempts to create a symlink to a file that does not exist (this was previously allowed on Linux/Mac). * One exception to this: when Spack installs files from the source into their final prefix, dangling symlinks are allowed (on Linux/Mac - Windows does not allow this in any circumstance). The intent behind this is to avoid generating failures for installations on Linux/Mac that were succeeding before. * Because Windows is strict about forbidding dangling symlinks, `traverse_tree` has been updated to skip creating symlinks if they would point to a file that is ignored. This check is not transitive (i.e., a symlink to a symlink to an ignored file would not be caught appropriately) * Relocate function: resolve_link_target_relative_to_the_link (this is not otherwise modified) Co-authored-by: jamessmillie <smillie@txcorp.com>
2023-06-26modules: ignore more Modules variables in from_sourcing_file (#38455)Xavier Delaruelle3-6/+51
Update list of excluded variables in `from_sourcing_file` function to cover all variables specific to Environment Modules or Lmod. Add specifically variables relative to the definition of `module()`, `ml()` and `_module_raw()` Bash functions. Fixes #13504
2023-06-26Add raw attribute to env.set command (#38465)Xavier Delaruelle4-9/+42
Update `env.set` command and underlying `SetEnv` object to add the `raw` boolean attribute. `raw` is optional and set to False by default. When set to True, value format is skipped for object when generating environment modifications. With this change it is now possible to define environment variable whose value contains variable reference syntax (like `{foo}` or `{}`) that should be set as-is. Fixes #29578
2023-06-23spack buildcache sync (--manifest-glob <glob> | <src> <dest>) are mutually ↵Harmen Stoppels1-3/+6
exclusive (#38533)
2023-06-18cc: Ensure that user-specified flags take precedence over others (#37376)Andrey Parfenov2-195/+256
Spack flags supplied by users should supersede flags from package build systems and other places in Spack. However, Spack currently adds user-supplied flags to the beginning of the compile line, which means that in some cases build system flags will supersede user-supplied ones. The right place to add a flag to ensure it has highest precedence for the compiler really depends on the type of flag. For example, search paths like `-L` and `-I` are examined in order, so adding them first is highest precedence. Compilers take the *last* occurrence of optimization flags like `-O2`, so those should be placed *after* other such flags. Shim libraries with `-l` should go *before* other libraries on the command line, so we want user-supplied libs to go first, etc. `lib/spack/env/cc` already knows how to split arguments into categories like `libs_list`, `rpath_dirs_list`, etc., so we can leverage that functionality to merge user flags into the arg list correctly. The general rules for injected flags are: 1. All `-L`, `-I`, `-isystem`, `-l`, and `*-rpath` flags from `spack_flags_*` to appear before their regular counterparts. 2. All other flags ordered with the ones from flags after their regular counterparts, i.e. `other_flags` before `spack_flags_other_flags` - [x] Generalize argument categorization into its own function in the `cc` shell script - [x] Apply the same splitting logic to injected flags and flags from the original compile line. - [x] Use the resulting flag lists to merge user- and build-system-supplied flags by category. - [x] Add tests. Signed-off-by: Andrey Parfenov <andrey.parfenov@intel.com> Co-authored-by: iermolae <igor.ermolaev@intel.com>
2023-06-18fix oneapi modules (#38400)simonleary-umass-edu1-1/+1
2023-06-16unparser: drop Python 2, fix testing bugs with newer Pythons (#38424)Greg Becker2-129/+20
The `unparser` that Spack uses for package hashing had several tweaks to ensure compatibility with Python 2.7: 1. Currently, the unparser automatically moves `*` and `**` args to the end to preserve compatibility with `python@:3.4` 2. `print a, b, c` statements and single-tuple `print((a, b, c))` function calls were remapped to `print(a, b, c)` in the unparsed output for consistency across versions. (1) is causing issues in our tests because a recent patch to the Python source code (https://github.com/python/cpython/pull/102953/files#diff-7972dffec6674d5f09410c71766ac6caacb95b9bccbf032061806ae304519c9bR813-R823) has a `**` arg before an named argument, and we round-trip the core python source code as a test of our unparser. This isn't actually a break with our consistent unpausing -- it's still consistent, the python source just doesn't unparse to the same thing anymore. It does makes it harder to test, so it's not worth maintaining the Python2-specific stuff anymore. Since we only support `python@3.6:`, this PR removes (1) and (2) from the unparser, but keeps one last tweak for unicode AST inconsistencies, as it's still needed for Python 3.5-3.7. This fixes the CI error we've been seeing on `python@3.11.4` and `python@3.10.12`. Again, that bug exists only in the test system and doesn't affect our canonical hashing of Python code.
2023-06-16show external status as [e] (#33792)Greg Becker2-14/+34
2023-06-16docs: add quotes around some values in a YAML example (#38412)Gurkirat Singh1-3/+3
2023-06-15Enable build system guess tests on windows (#36971)markus-ferrell2-5/+1
2023-06-15Windows testing: enable graph, mark, and info cmd tests (#36977)markus-ferrell3-11/+0
2023-06-15Add virtual information on DAG edges (#34821)Massimiliano Culpo18-136/+290
* DependencySpec: add virtuals attribute on edges This works for both the new and the old concretizer. Also, added type hints to involved functions. * Improve virtual reconstruction from old format * Reconstruct virtuals when reading from Cray manifest * Reconstruct virtual information on test dependencies
2023-06-14modules: use depends-on to autoload module with Lmod on Tcl (#38347)Xavier Delaruelle1-0/+27
Update Tcl modulefile template to use the `depends-on` command to autoload modules if Lmod is the current module tool. Autoloading modules with `module load` command in Tcl modulefile does not work well for Lmod at some extend. An attempt to unload then load designated module is performed each time such command is encountered. It may lead to a load storm that may not end correctly with large number of module dependencies. `depends-on` command should be used for Lmod instead of `module load`, as it checks if module is already loaded, and does not attempt to reload this module. Lua modulefile template already uses `depends_on` command to autoload dependencies. Thus it is already considered that to use Lmod with Spack, it must support `depends_on` command (version 7.6+). Environment Modules copes well with `module load` command to autoload dependencies (version 3.2+). `depends-on` command is supported starting version 5.1 (as an alias of `prereq-all` command) which was relased last year. This change introduces a test to determine if current module tool that evaluates modulefile is Lmod. If so, autoload dependencies are defined with `depends-on` command. Otherwise `module load` command is used. Test is based on `LMOD_VERSION_MAJOR` environment variable, which is set by Lmod starting version 5.1. Fixes #36764
2023-06-14build(deps): bump python-levenshtein in /lib/spack/docs (#38339)dependabot[bot]1-1/+1
Bumps [python-levenshtein](https://github.com/maxbachmann/python-Levenshtein) from 0.21.0 to 0.21.1. - [Release notes](https://github.com/maxbachmann/python-Levenshtein/releases) - [Changelog](https://github.com/maxbachmann/python-Levenshtein/blob/main/HISTORY.md) - [Commits](https://github.com/maxbachmann/python-Levenshtein/compare/v0.21.0...v0.21.1) --- updated-dependencies: - dependency-name: python-levenshtein dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-13Windows bugfix: path-to-URL conversion (#37827)Dan Lipsa2-12/+5
When interpreting local paths as relative URL endpoints, they were formatted as Windows paths on Windows (i.e. with '\'). URLs should always be POSIX-style.
2023-06-13modules: append trailing delimiter to MANPATH when set (#36678)Xavier Delaruelle3-1/+86
Update modulefile templates to append a trailing delimiter to MANPATH environment variable, if the modulefile sets it. With a trailing delimiter at ends of MANPATH's value, man will search the system man pages after searching the specific paths set. Using append-path/append_path to add this element, the module tool ensures it is appended only once. When modulefile is unloaded, the number of append attempt is decreased, thus the trailing delimiter is removed only if this number equals 0. Disclaimer: no path element should be appended to MANPATH by generated modulefiles. It should always be prepended to ensure this variable's value ends with the trailing delimiter. Fixes #11355.
2023-06-12pip is a pythonextension not a pythonpackage, and it turns out we werent ↵Peter Scheibel2-47/+46
doing our external surgery on things that inherited pythonextension (#38186)
2023-06-12buildcache: remove deprecated api (#37246)Harmen Stoppels10-347/+66
The API was deprecated in v0.20 and is slated for removal in v0.21
2023-06-12bootstrap: add minimum versions required for flake8 and pytest (#38322)Massimiliano Culpo1-2/+2
2023-06-11Fix typo in example code block (#38318)Christopher Christofi1-1/+1
2023-06-11build(deps): bump urllib3 from 2.0.2 to 2.0.3 in /lib/spack/docs (#38244)dependabot[bot]1-1/+1
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.0.2 to 2.0.3. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](https://github.com/urllib3/urllib3/compare/2.0.2...2.0.3) --- updated-dependencies: - dependency-name: urllib3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-11build(deps): bump sphinx-rtd-theme in /lib/spack/docs (#38245)dependabot[bot]1-1/+1
Bumps [sphinx-rtd-theme](https://github.com/readthedocs/sphinx_rtd_theme) from 1.2.1 to 1.2.2. - [Changelog](https://github.com/readthedocs/sphinx_rtd_theme/blob/master/docs/changelog.rst) - [Commits](https://github.com/readthedocs/sphinx_rtd_theme/compare/1.2.1...1.2.2) --- updated-dependencies: - dependency-name: sphinx-rtd-theme dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-09install status arg: update to default true and add inverse (#38252)Greg Becker4-5/+24
* install status arg: update to default true and add inverse * update bash completion * fix tests broken by output change
2023-06-08Forbid using `env:` as a top level environment attribute (#38199)Massimiliano Culpo10-80/+60
* Remove "env" from environment schema * Remove spack.env.schema.keys * Remove spack.environment.config_dict
2023-06-08CMake/Windows bugfix: Make CMAKE_INSTALL_PREFIX a posix path (#36842)John W. Parent1-2/+2
CMake gives off a warning when passed Windows style paths as install prefixes as the resultant path often causes invalid escape sequences.
2023-06-08Fix compiler removal from command line (#38057)Massimiliano Culpo9-251/+200
* Improve lib/spack/spack/test/cmd/compiler.py * Use "tmp_path" in the "mock_executable" fixture * Return a pathlib.Path from mock_executable * Fix mock_executable fixture on Windows "mock_gcc" was very similar to mock_executable, so use the latter to reduce code duplication * Remove wrong compiler cache, fix compiler removal fixes #37996 _CACHE_CONFIG_FILES was both unneeded and wrong, if called subsequently with different scopes. Here we remove that cache, and we fix an issue with compiler removal triggered by having the same compiler spec in multiple scopes.
2023-06-07Add documentation about package signing model (#30939)Christopher Kotfila2-0/+485
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com> Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-06-06E4S Cray CI Stack (#37837)eugeneswalker1-1/+1
* e4s cray ci stack * e4s ci: add cray * add zen4 tag * WIP: new defintions just for cray * updates * remove ci signing job overrride, not necessary * echo $PATH and show modules loaded * add mirror * add external def for cray-libsci * comment out quantum-espresso * use /etc/protected-runner as key path * cray ci stack: do not remove tags: [spack, public] * make cray stack composable * generate job should run on public tagged runner, override default config:install_tree:root * CI: Use relative path in default script * CI: Use relative includes paths for shell runners * Use concrete_env_dir for relpath * ml-darwin-aarch64-mps: jax has bazel codesign issue --------- Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com> Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2023-06-06CachedCMakePackage: fix bug where CMAKE_CUDA_ARCHITECTURES=none is set (#38169)Mikael Simberg1-2/+2
#37592 updated cached cmake packages to set CMAKE_CUDA_ARCHITECTURES. The condition `if archs != "none"` lead to `CMAKE_CUDA_ARCHITECTURES=none` when cuda_arch=none (incorrect check on the value of a multi-valued variant), i.e. CMAKE_CUDA_ARCHITECTURES is always set. This PR udpates the condition to if archs[0] != "none" to ensure CMAKE_CUDA_ARCHITECTURES is only set if cuda_arch is not none (which seems to be the pattern used in other packages). This does the same for HIP (although in general ROCmPackage disallows amdgpu_target=none when +rocm).
2023-06-06containers: don't install epel-release on Fedora (#37766)Jonathon Anderson2-8/+18
2023-06-05Update RtD and Sphinx configuration (#38046)Massimiliano Culpo3-42/+29
2023-06-01Memoize a few hot functions during module file generation (#37739)Massimiliano Culpo5-13/+16
2023-06-01Spec.format: print false attributes if requested (#37932)Greg Becker2-1/+2
2023-05-31CachedCMakePackage: add CUDA/HIP options and improve independent builds (#37592)Brian Van Essen1-9/+83
* Add CMake options for building with CUDA/HIP support to CachedCMakePackages (intended to reduce duplication across packages building with +hip/+cuda and using CachedCMakePackage) * Define generic variables like CMAKE_PREFIX_PATH for CachedCMakePackages (so that a user may invoke "cmake" themselves without needing to setthem on the command line). * Make `lbann` a CachedCMakePackage. Co-authored-by: Chris White <white238@llnl.gov>
2023-05-30Bugfix: cray manifest parsing regression (#37909)Tiziano Müller3-4/+18
fa7719a changed syntax for specifying exact versions, which are required for some compiler specs (including those read as part of parsing a Cray manifest). This fixes that and also makes a couple other improvements to manifest parsing. * Instantiate compiler specs with exact versions (fixes #37893) * fix slingshot network detection (CPE 22.10+ has libcxi.so in /usr/lib64) * "spack external find": add arg to ignore default dir for cray manifests
2023-05-30Simplify implementation of "get_compiler_config" (#37989)Massimiliano Culpo1-28/+18
2023-05-29Remove patchelf self-relocation (#33834)Harmen Stoppels2-43/+1
2023-05-29fix InternalConcretizerError msg (#37791)Harmen Stoppels1-6/+7
2023-05-29Bugfix/tests: add slash to test log message (#37874)Tamara Dahlgren1-1/+1
2023-05-29tests/sip: convert to new stand-alone test process (#35693)Tamara Dahlgren1-7/+10
2023-05-28modules: improve default naming scheme (#37808)Xavier Delaruelle4-38/+38
Change default naming scheme for tcl modules for a more user-friendly experience. Change from flat projection to "per software name" projection. Flat naming scheme restrains module selection capabilities. The `{name}/{version}...` scheme make possible to use user-friendly mechanisms: * implicit defaults (`module load git`) * extended default (`module load git/2`) * advanced version specifiers (`module load git@2:`)