summaryrefslogtreecommitdiff
path: root/etc
AgeCommit message (Collapse)AuthorFilesLines
2023-11-08Revert "defaults/modules.yaml: hide implicits (#40906)" (#40955)Harmen Stoppels1-2/+0
This reverts commit a2f00886e911a8219bfac27752e5c7fd83c65280.
2023-11-06Add support for aliases (#17229)Michael Kuhn1-0/+8
Add a new config section: `config:aliases`, which is a dictionary mapping aliases to commands. For instance: ```yaml config: aliases: sp: spec -I ``` will define a new command `sp` that will execute `spec` with the `-I` argument. Aliases cannot override existing commands, and this is ensured with a test. We cannot currently alias subcommands. Spack will warn about any aliases containing a space, but will not error, which leaves room for subcommand aliases in the future. --------- Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2023-11-06defaults/modules.yaml: hide implicits (#40906)Harmen Stoppels1-0/+2
2023-10-11Update bootstrap buildcache to support Python 3.12 (#40404)Massimiliano Culpo1-3/+3
* Add support for Python 3.12 * Use optimized build of clingo
2023-10-06Make "minimal" the default duplicate strategy (#39621)Massimiliano Culpo1-1/+1
* Allow branching out of the "generic build" unification set For cases like the one in https://github.com/spack/spack/pull/39661 we need to relax rules on unification sets. The issue is that, right now, nodes in the "generic build" unification set are unified together with their build dependencies. This was done out of caution to avoid the risk of circular dependencies, which would ultimately cause a very slow solve. For build-tools like Cython, however, the build dependencies is masked by a long chain of "build, run" dependencies that belong in the "generic build" unification space. To allow splitting on cases like this, we relax the rule disallowing branching out of the "generic build" unification set. * Fix issue with pure build virtual dependencies Pure build virtual dependencies were not accounted properly in the list of possible virtuals. This caused some facts connecting virtuals to the corresponding providers to not be emitted, and in the end lead to unsat problems. * Fixed a few issues in packages py-gevent: restore dependency on py-cython@3 jsoncpp: fix typo in build dependency ecp-data-vis-sdk: update spack.yaml and cmake recipe py-statsmodels: add v0.13.5 * Make dependency on "blt" of type "build"
2023-08-17`zlib-api`: use `zlib-ng +compat` by default (#39358)Harmen Stoppels1-1/+1
In the HPC package manager, we want the fastest `zlib` implementation by default. `zlib-ng` is up to 4x faster than stock `zlib`, and it can do things like take advantage of AVX-512 instructions. This PR makes `zlib-ng` the default `zlib-api` provider (`zlib-api` was introduced earlier, in #37372). As far as I can see, the only issues you can encounter are: 1. Build issues with packages that heavily rely on `zlib` internals. In Gitlab CI only one out of hundreds of packages had that issue (it extended zlib with deflate stuff, and used its own copy of zlib sources). 2. Packages that like to detect `zlib-ng` separately and rely on `zlib-ng` internals. The only issue I've found with this among the hundreds of packages built in CI is `perl` trying to report more specific zlib-ng version details, and relied on some internals that got refactored. But yeah... that warrants a patch / conflict and is nothing special. At runtime, you cannot really have any issues, given that zlib and zlib-ng export the exact same symbols (and zlib-ng tests this in their CI). You can't really have issues with externals when using zlib-ng either. The only type of issue is when system zlib is rather new, and not marked as external; if another external uses new symbols, and Spack builds an older zlib/zlib-ng, then the external might not find the new symbols. But this is a configuration issue, and it's not an issue caused by zlib-ng, as the same would happen with older Spack zlib. * zlib-api: use zlib-ng +compat by default * make a trivial change to zlib-ng to trigger a rebuild * add `haampie` as maintainer
2023-08-15Add a new configuration option to select among different concretization modesMassimiliano Culpo1-0/+6
The "concretizer" section has been extended with a "duplicates:strategy" attribute, that can take three values: - "none": only 1 node per package - "minimal": allow multiple nodes opf specific packages - "full": allow full duplication for a build tool
2023-08-09zlib-api: new virtual with zlib/zlib-ng as providers (#37372)Harmen Stoppels1-0/+1
Introduces a new virtual zlib-api, which replaces zlib in most packages. This allows users to switch to zlib-ng by default for better performance.
2023-08-02Add qmake virtual provider (#38848)Adam J. Stewart1-0/+1
2023-07-13mirrors: distinguish between source/binary mirror; simplify schema (#34523)Harmen Stoppels1-1/+3
Allow the following formats: ```yaml mirrors: name: <url> ``` ```yaml mirrors: name: url: s3://xyz access_pair: [x, y] ``` ```yaml mirrors: name: fetch: http://xyz push: url: s3://xyz access_pair: [x, y] ``` And reserve two new properties to indicate the mirror type (e.g. mirror.spack.io is a source mirror, not a binary cache) ```yaml mirrors: spack-public: source: true binary: false url: https://mirror.spack.io ```
2023-07-12installer: Improve status reporting (#37903)Michael Kuhn1-4/+5
Refactor `TermTitle` into `InstallStatus` and use it to show progress information both in the terminal title as well as inline. This also turns on the terminal title status by default. The inline output will look like the following after this change: ``` ==> Installing m4-1.4.19-w2fxrpuz64zdq63woprqfxxzc3tzu7p3 [4/4] ```
2023-05-11Update providers of virtual packages related to Intel OneAPI (#37412)Nisarg Patel1-3/+3
* add a virtual dependency name instead of complete package name * add OneAPI components as providers of virtual packages * Revert the default of tbb --------- Co-authored-by: Nisarg Patel <nisarg.patel@lrz.de>
2023-05-02Disable module file generation by default (#37258)Massimiliano Culpo1-3/+2
* Disable module generation by default (#35564) a) It's used by site administrators, so it's niche b) If it's used by site administrators, they likely need to modify the config anyhow, so the default config only serves as an example to get started c) it's too arbitrary to enable tcl, but disable lmod * Remove leftover from old module file schema * Warn if module file config is detected and generation is disabled --------- Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2023-04-27macOS: use Apple GL/GLU by default (#36618)Adam J. Stewart1-0/+12
* macOS: use Apple GL/GLU by default * Use CLT instead * Use CLT instead * Undo change to libuuid
2023-03-31Allow configurable stage names (#36509)John W. Parent1-0/+1
Add `config:stage_name` which is a Spec format string that can customize the names of stages created by Spack. This was primarily created to allow generating shorter stage names on Windows (along with `config:build_stage`, this can be used to create stages with short absolute paths). By default, this is not set and the prior name stage format is used. This also removes the username component that is always added to Stage paths on Windows (if users want to include this, they can add it to the `build_stage`).
2023-03-28WGL package: correct libs/headers detection (#35113)John W. Parent1-0/+1
Corrects libs detection with a more specific root, otherwise there can be inconsistencies between version of WGL requested and the version picked up by `find_libraries`. Corrects headers detection - win-sdk, win-wdk, and WGL headers all exist under the same directory, so we can compute the headers for WGL without querying the spec for win-sdk (which causes errors). This commit also removes the `plat` variant of `wgl`, which is redundant with the Spec's target.
2023-03-23modules tcl: switch default all:autoload from none to direct (#36269)Xavier Delaruelle1-1/+1
Since environment-modules has support for autoloading since 4.2, and Spack-builds of it enable it by default, use the same autoload default for tcl as lmod.
2023-03-14concretizer: add mode to reuse dependencies only (#30990)Michael Kuhn1-2/+3
This adds a new mode for `concretizer:reuse` called `dependencies`, which only reuses dependencies. Currently, `spack install foo` will reuse older versions of `foo`, which might be surprising to users.
2023-03-10go: refactor bootstrapping process (#35823)Alec Scott1-1/+1
* Refactor go bootstrapping to include binary or gcc bootstrap
2023-03-09concretizer.yaml: document valid values for granularity (#35961)Harshula Jayasuriya1-4/+5
2023-02-22Allow users to specify root env dir (#32836)psakievich1-0/+4
* Allow users to specify root env dir Environments managed by spack have some advantages over anonymous Environments but they are tucked away inside spack's directory tree. This PR gives users the ability to specify where the environments should live. See #32823 This is also taken as an opportunity to ensure that all references are to "managed environments", rather than "named environments". Prior to this PR some references to the latter persisted. Co-authored-by: Tom Scogland <scogland1@llnl.gov> Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com> Co-authored-by: Gregory Becker <becker33@llnl.gov>
2023-02-21Revert "Disable module generation by default (#35564)" (#35603)Massimiliano Culpo1-2/+3
This reverts commit 5b8917188a4d84713c1037fd9464bf77503edd27.
2023-02-21Disable module generation by default (#35564)Harmen Stoppels1-3/+2
a) It's used by site administrators, so it's niche b) If it's used by site administrators, they likely need to modify the config anyhow, so the default config only serves as an example to get started c) it's too arbitrary to enable tcl, but disable lmod
2023-02-17Increase db timeout 3s -> 60s (#35517)Harmen Stoppels1-1/+1
When running many concurrent spack install processes that need to write to the db, Spack regularly times out. This is because writing to the DB after another process has written to it requires deserialization of the db, mutating it in memory, and serializing it again, which takes some time. On top of that, I believe there's a 1 second retry when a write lock cannot be obtained, so I think this means only 3 processes can really write to the DB at the same time before timing out.
2023-01-17Doc: config.yaml mention $env (#34905)Axel Huebl1-0/+5
Add one more note on the $env variable in `config.yaml`.
2023-01-03Windows: package defaults and MPI detection (#34614)John W. Parent1-0/+21
* Update packages config to indicate that MSVC is the preferred compiler * Update packages config to indicate that msmpi is the preferred MPI provider * Fix msmpi external detection
2022-11-28warn about removal of deprecated format strings (#34101)Greg Becker1-2/+1
* warn about removal of deprecated format strings Co-authored-by: becker33 <becker33@users.noreply.github.com>
2022-11-23Control Werror by converting to Wno-error (#30882)Tom Scogland1-1/+6
Using `-Werror` is good practice for development and testing, but causes us a great deal of heartburn supporting multiple compiler versions, especially as newer compiler versions add warnings for released packages. This PR adds support for suppressing `-Werror` through spack's compiler wrappers. There are currently three modes for the `flags:keep_werror` setting: * `none`: (default) cancel all `-Werror`, `-Werror=*` and `-Werror-*` flags by converting them to `-Wno-error[=]*` flags * `specific`: preserve explicitly selected warnings as errors, such as `-Werror=format-truncation`, but reverse the blanket `-Werror` * `all`: keeps all `-Werror` flags These can be set globally in config.yaml, through the config command-line flags, or overridden by a particular package (some packages use Werror as a proxy for determining support for other compiler features). We chose to use this approach because: 1. removing `-Werror` flags entirely broke *many* build systems, especially autoconf based ones, because of things like checking `-Werror=feature` and making the assumption that if that did not error other flags related to that feature would also work 2. Attempting to preserve `-Werror` in some phases but not others caused similar issues 3. The per-package setting came about because some packages, even with all these protections, still use `-Werror` unsafely. Currently there are roughly 3 such packages known.
2022-11-22Revert "Warn about removal of deprecated format strings (#33829)" (#34056)Massimiliano Culpo1-2/+2
This reverts commit 7f9af8d4a0bfbb1577e5ac9982624d8d0cb9b9ca.
2022-11-22Warn about removal of deprecated format strings (#33829)Greg Becker1-2/+2
Co-authored-by: becker33 <becker33@users.noreply.github.com>
2022-11-07concretizer:unify:true as a default (#31787)Harmen Stoppels1-1/+1
`spack env create` enables a view by default (in a weird hidden directory, but well...). This is asking for trouble with the other default of `concretizer:unify:false`, since having different flavors of the same spec in an environment, leads to collision errors when generating the view. A change of defaults would improve user experience: However, `unify:true` makes most sense, since any time the issue is brought up in Slack, the user changes the concretization config, since it wasn't the intention to have different flavors of the same spec, and install times are decreased. Further we improve the docs and drop the duplicate root spec limitation
2022-11-03Experimental binding of shared ELF libraries (#31948)Harmen Stoppels1-4/+14
Adds another post install hook that loops over the install prefix, looking for shared libraries type of ELF files, and sets the soname to their own absolute paths. The idea being, whenever somebody links against those libraries, the linker copies the soname (which is the absolute path to the library) as a "needed" library, so that at runtime the dynamic loader realizes the needed library is a path which should be loaded directly without searching. As a result: 1. rpaths are not used for the fixed/static list of needed libraries in the dynamic section (only for _actually_ dynamically loaded libraries through `dlopen`), which largely solves the issue that Spack's rpaths are a heuristic (`<prefix>/lib` and `<prefix>/lib64` might not be where libraries really are...) 2. improved startup times (no library search required)
2022-11-03Update metadata for bootstrapping (#33665)Massimiliano Culpo1-4/+3
2022-10-31Windows: fix library loading and enable Clingo bootstrapping (#33400)John W. Parent1-1/+1
Changes to improve locating shared libraries on Windows, which in turn enables the use of Clingo. This PR attempts to establish a proper distinction between linking on Windows vs. Linux/Mac: on Windows, linking is always done with .lib files (never .dll files). This somewhat complicates the model since the Spec.lib method could return libraries that were used for both linking and loading, but since these are not always the same on Windows, it was decided to treat Spec.libs as being for link-time libraries. Additional functions are added to help dependents locate run-time libraries. * Clingo is now the default concretizer on Windows * Clingo is now the concretizer used for unit tests on Windows * Fix a permissions issue that can occur while moving Git files during fetching/staging * Packages can now implement "win_add_library_dependent" to register files/directories that include libraries that would need to link to dependency dlls * Packages can now implement "win_add_rpath" to register the locations of dlls that dependents would want to load * "Spec.libs" on Windows is updated to return link-time libraries (i.e. .lib files, rather than .dll files) * PackageBase.rpath on Windows is now updated to return the most-likely locations where .dlls will be found (which is generally in the bin/ directory)
2022-10-19Update the binary index before attempting direct fetches (#32137)Jonathon Anderson1-0/+4
"spack install" will not update the binary index if given a concrete spec, which causes it to fall back to direct fetches when a simple index update would have helped. For S3 buckets in particular, this significantly and needlessly slows down the install process. This commit alters the logic so that the binary index is updated whenever a by-hash lookup fails. The lookup is attempted again with the updated index before falling back to direct fetches. To avoid updating too frequently (potentially once for each spec being installed), BinaryCacheIndex.update now includes a "cooldown" option, and when this option is enabled it will not update more than once in a cooldown window (set in config.yaml). Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2022-10-18go,gcc: Support external go compilers for Go bootstrap (#27769)Bernhard Kaindl1-1/+2
For ARM64, fallback to gccgo. (go-bootstrap@1.4 can't support ARM64)
2022-09-02Update bootstrap buildcache to v0.3 (#32262)Harmen Stoppels1-1/+3
This release allow to bootstrap patchelf from binaries.
2022-08-23Relative paths in default prefix_inspections start with ./ (#31867)Harmen Stoppels2-11/+12
2022-08-11Don't set `LD_LIBRARY_PATH` by default on Linux (#28354)Harmen Stoppels2-12/+2
`LD_LIBRARY_PATH` can break system executables (e.g., when an enviornment is loaded) and isn't necessary thanks to `RPATH`s. Packages that require `LD_LIBRARY_PATH` can set this in `setup_run_environment`. - [x] Prefix inspections no longer set `LD_LIBRARY_PATH` by default - [x] Document changes and workarounds for people who want `LD_LIBRARY_PATH`
2022-07-29OpenGL restructure (#31748)Chuck Atkins1-4/+4
* OpenGL: Restructures the OpenGL packages This provides concrete glx and osmesa packages which delegate to virtual libglx and libosmesa packages provided by mesa. This was necessary because GLX and OSMesa both provide gl implementations but with mesa providing the girtual gl package there was no way to properly distinguish which of the two opengl implementations was beiing requested when querying the spec['gl'] dependency. This additional level of indirection allows for that. * OpenGL: Adjust downstream dependents of OpenGL for the restructure This implements the necessary fixes in the packages that depend on OpenGL to work with the restructuring. This also attempts to create a consistent variant for specifying glx or osmesa.
2022-05-24Add a command to generate a local mirror for bootstrapping (#28556)Massimiliano Culpo1-24/+5
This PR builds on #28392 by adding a convenience command to create a local mirror that can be used to bootstrap Spack. This is to overcome the inconvenience in setting up this mirror manually, which has been reported when trying to setup Spack on air-gapped systems. Using this PR the user can create a bootstrapping mirror, on a machine with internet access, by: % spack bootstrap mirror --binary-packages /opt/bootstrap ==> Adding "clingo-bootstrap@spack+python %apple-clang target=x86_64" and dependencies to the mirror at /opt/bootstrap/local-mirror ==> Adding "gnupg@2.3: %apple-clang target=x86_64" and dependencies to the mirror at /opt/bootstrap/local-mirror ==> Adding "patchelf@0.13.1:0.13.99 %apple-clang target=x86_64" and dependencies to the mirror at /opt/bootstrap/local-mirror ==> Adding binary packages from "https://github.com/alalazo/spack-bootstrap-mirrors/releases/download/v0.1-rc.2/bootstrap-buildcache.tar.gz" to the mirror at /opt/bootstrap/local-mirror To register the mirror on the platform where it's supposed to be used run the following command(s): % spack bootstrap add --trust local-sources /opt/bootstrap/metadata/sources % spack bootstrap add --trust local-binaries /opt/bootstrap/metadata/binaries The mirror has to be moved over to the air-gapped system, and registered using the commands shown at prompt. The command has options to: 1. Add pre-built binaries downloaded from Github (default is not to add them) 2. Add development dependencies for Spack (currently the Python packages needed to use spack style) * bootstrap: refactor bootstrap.yaml to move sources metadata out * bootstrap: allow adding/removing custom bootstrapping sources This operation can be performed from the command line since new subcommands have been added to `spack bootstrap` * Add --trust argument to spack bootstrap add * Add a command to generate a local mirror for bootstrapping * Add a unit test for mirror creation
2022-05-23Deprecate `spack:concretization` over `concretizer:unify` (#30038)Harmen Stoppels1-0/+6
* Introduce concretizer:unify option to replace spack:concretization * Deprecate concretization * Make spack:concretization overrule concretize:unify for now * Add environment update logic to move from spack:concretization to spack:concretizer:reuse * Migrate spack:concretization to spack:concretize:unify in all locations * For new environments make concretizer:unify explicit, so that defaults can be changed in 0.19
2022-05-18Add license dir to config (#30135)robgics1-0/+3
* Change license dir from hard-coded to a configurable item * Change config item to be a string not an array Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2022-05-13Reuse concretization by default (#30396)Massimiliano Culpo1-2/+2
* Enable reuse by default in Spack * Update documentation to match new default * Configure pipelines not to reuse software
2022-05-10Add a Lua build-system (#28854)Tom Scogland1-1/+2
Reworking lua to allow easier substitution of the base lua implementation. Also adding in a maintained version of luajit and re-factoring the entire stack to use a custom build-system to centralize functionality like environment variable management and luarocks installation. The `lua-lang` virtual is now versioned so that a package that requires Lua 5.1 semantics can get any lua, but one that requires 5.2 will only get upstream lua. The luaposix package requires lua-bit32, but only when built with a lua conforming to version 5.1. This adds the package, and the dependencies, but exposed a problem with luarocks dependency detection. Since we're installing each package in its own "tree" and there's no environment variable to list extra trees, spack now generates a luarocks config file that lists all the trees of all the dependencies, and references it by setting `LUAROCKS_CONFIG` in the build environment of every LuaPackage. This allows luarocks to find the spack installed dependencies correctly rather than trying (and failing) to download them. Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Tom Scogland <tscogland@llnl.gov> Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-04-25ASP-based solver: allow configuring target selection (#29835)Massimiliano Culpo1-0/+13
* ASP-based solver: allow configuring target selection This commit adds a new "concretizer:targets" configuration section, and two options under it. - "concretizer:targets:granularity" allows switching from considering only generic targets to consider all possible microarchitectures. - "concretizer:targets:host_compatible" instead controls whether we can concretize for microarchitectures that are incompatible with the current host. * Add documentation * Add unit-tests
2022-04-13Add support for Python 3.10 (#29581)Massimiliano Culpo1-4/+13
* Add support for Python 3.10 * Update unit-tests to use 3.10 * Update Getting started section of the docs * Update bootstrap action
2022-04-08Deprecate top-level module config (#28659)Harmen Stoppels1-6/+3
* Ignore top-level module config; add auto-update In Spack 0.17 we got module sets (modules:[name]:[prop]), and for backwards compat modules:[prop] was short for modules:default:[prop]. But this makes it awkward to define default config for the "default" module set. Since 0.17 is branched off, we can now deprecate top-level module config (that is, just ignore it with a warning). This PR does that, and it implements `spack config update modules` to make upgrading easy (we should have added that to 0.17 already...) It also removes references to `dotkit` stuff which was already deprecated in 0.13 and could have been removed in 0.14. Prefix inspections are the only exception, since the top-level prefix inspections used for `spack load` and `spack env activate`.
2022-03-17Expand external find for Windows (#27588)John Parent1-1/+1
* Incorporate new search location * Add external user option * proper doc string * Explicit commands in getting started * raise during chgrp on Win recover installer changes Notate admin privleges Windows phase install hooks Find external python and install ninja (#23496) Allow external find python to find windows python and spack install ninja Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17Packaging of netlib-lapack for windows (#24993)Jared Popelar1-0/+5
MSVC's internal CMake and Ninja now detected by spack external find and added to packages.yaml Saving progress on packaging zlib for Windows Fixing the shared CMake flag * Loading Intel's ifx Fortran compiler into MSVC; if there are multiple versions of MSVC installed and detected, ifx will only be placed into the first block written in compilers.yaml. The version number of ifx can be detected using MSVC's version flag (instead of /QV) by using ignore_version_errors. This commit also provides support for detection of Intel compilers in their own compiler block by adding ifx.exe to the fc/f77_name blocks inside intel.py * Giving CMake a Fortran compiler argument * Adding patch file for removing duplicated mangling header for versions 3.9.1 and older; static and shared now successfully building on Windows * Have netlib-lapack depend on ninja@1.10 Co-authored-by: John R. Cary <cary@txcorp.com> Co-authored-by: Jared Popelar <jpopelar@txcorp.com> Making a default config.yaml for Windows Small path length for build_stage Provide more prerequisite details, mention default config.yaml Killing an unnecessary setvars call Replacing some lost changes, proofreading, updating windows-supported package list Co-authored-by: John Parent <john.parent@kitware.com>