summaryrefslogtreecommitdiff
path: root/etc
AgeCommit message (Collapse)AuthorFilesLines
2022-11-23Control Werror by converting to Wno-error (#30882)Tom Scogland1-1/+6
Using `-Werror` is good practice for development and testing, but causes us a great deal of heartburn supporting multiple compiler versions, especially as newer compiler versions add warnings for released packages. This PR adds support for suppressing `-Werror` through spack's compiler wrappers. There are currently three modes for the `flags:keep_werror` setting: * `none`: (default) cancel all `-Werror`, `-Werror=*` and `-Werror-*` flags by converting them to `-Wno-error[=]*` flags * `specific`: preserve explicitly selected warnings as errors, such as `-Werror=format-truncation`, but reverse the blanket `-Werror` * `all`: keeps all `-Werror` flags These can be set globally in config.yaml, through the config command-line flags, or overridden by a particular package (some packages use Werror as a proxy for determining support for other compiler features). We chose to use this approach because: 1. removing `-Werror` flags entirely broke *many* build systems, especially autoconf based ones, because of things like checking `-Werror=feature` and making the assumption that if that did not error other flags related to that feature would also work 2. Attempting to preserve `-Werror` in some phases but not others caused similar issues 3. The per-package setting came about because some packages, even with all these protections, still use `-Werror` unsafely. Currently there are roughly 3 such packages known.
2022-11-22Revert "Warn about removal of deprecated format strings (#33829)" (#34056)Massimiliano Culpo1-2/+2
This reverts commit 7f9af8d4a0bfbb1577e5ac9982624d8d0cb9b9ca.
2022-11-22Warn about removal of deprecated format strings (#33829)Greg Becker1-2/+2
Co-authored-by: becker33 <becker33@users.noreply.github.com>
2022-11-07concretizer:unify:true as a default (#31787)Harmen Stoppels1-1/+1
`spack env create` enables a view by default (in a weird hidden directory, but well...). This is asking for trouble with the other default of `concretizer:unify:false`, since having different flavors of the same spec in an environment, leads to collision errors when generating the view. A change of defaults would improve user experience: However, `unify:true` makes most sense, since any time the issue is brought up in Slack, the user changes the concretization config, since it wasn't the intention to have different flavors of the same spec, and install times are decreased. Further we improve the docs and drop the duplicate root spec limitation
2022-11-03Experimental binding of shared ELF libraries (#31948)Harmen Stoppels1-4/+14
Adds another post install hook that loops over the install prefix, looking for shared libraries type of ELF files, and sets the soname to their own absolute paths. The idea being, whenever somebody links against those libraries, the linker copies the soname (which is the absolute path to the library) as a "needed" library, so that at runtime the dynamic loader realizes the needed library is a path which should be loaded directly without searching. As a result: 1. rpaths are not used for the fixed/static list of needed libraries in the dynamic section (only for _actually_ dynamically loaded libraries through `dlopen`), which largely solves the issue that Spack's rpaths are a heuristic (`<prefix>/lib` and `<prefix>/lib64` might not be where libraries really are...) 2. improved startup times (no library search required)
2022-11-03Update metadata for bootstrapping (#33665)Massimiliano Culpo1-4/+3
2022-10-31Windows: fix library loading and enable Clingo bootstrapping (#33400)John W. Parent1-1/+1
Changes to improve locating shared libraries on Windows, which in turn enables the use of Clingo. This PR attempts to establish a proper distinction between linking on Windows vs. Linux/Mac: on Windows, linking is always done with .lib files (never .dll files). This somewhat complicates the model since the Spec.lib method could return libraries that were used for both linking and loading, but since these are not always the same on Windows, it was decided to treat Spec.libs as being for link-time libraries. Additional functions are added to help dependents locate run-time libraries. * Clingo is now the default concretizer on Windows * Clingo is now the concretizer used for unit tests on Windows * Fix a permissions issue that can occur while moving Git files during fetching/staging * Packages can now implement "win_add_library_dependent" to register files/directories that include libraries that would need to link to dependency dlls * Packages can now implement "win_add_rpath" to register the locations of dlls that dependents would want to load * "Spec.libs" on Windows is updated to return link-time libraries (i.e. .lib files, rather than .dll files) * PackageBase.rpath on Windows is now updated to return the most-likely locations where .dlls will be found (which is generally in the bin/ directory)
2022-10-19Update the binary index before attempting direct fetches (#32137)Jonathon Anderson1-0/+4
"spack install" will not update the binary index if given a concrete spec, which causes it to fall back to direct fetches when a simple index update would have helped. For S3 buckets in particular, this significantly and needlessly slows down the install process. This commit alters the logic so that the binary index is updated whenever a by-hash lookup fails. The lookup is attempted again with the updated index before falling back to direct fetches. To avoid updating too frequently (potentially once for each spec being installed), BinaryCacheIndex.update now includes a "cooldown" option, and when this option is enabled it will not update more than once in a cooldown window (set in config.yaml). Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2022-10-18go,gcc: Support external go compilers for Go bootstrap (#27769)Bernhard Kaindl1-1/+2
For ARM64, fallback to gccgo. (go-bootstrap@1.4 can't support ARM64)
2022-09-02Update bootstrap buildcache to v0.3 (#32262)Harmen Stoppels1-1/+3
This release allow to bootstrap patchelf from binaries.
2022-08-23Relative paths in default prefix_inspections start with ./ (#31867)Harmen Stoppels2-11/+12
2022-08-11Don't set `LD_LIBRARY_PATH` by default on Linux (#28354)Harmen Stoppels2-12/+2
`LD_LIBRARY_PATH` can break system executables (e.g., when an enviornment is loaded) and isn't necessary thanks to `RPATH`s. Packages that require `LD_LIBRARY_PATH` can set this in `setup_run_environment`. - [x] Prefix inspections no longer set `LD_LIBRARY_PATH` by default - [x] Document changes and workarounds for people who want `LD_LIBRARY_PATH`
2022-07-29OpenGL restructure (#31748)Chuck Atkins1-4/+4
* OpenGL: Restructures the OpenGL packages This provides concrete glx and osmesa packages which delegate to virtual libglx and libosmesa packages provided by mesa. This was necessary because GLX and OSMesa both provide gl implementations but with mesa providing the girtual gl package there was no way to properly distinguish which of the two opengl implementations was beiing requested when querying the spec['gl'] dependency. This additional level of indirection allows for that. * OpenGL: Adjust downstream dependents of OpenGL for the restructure This implements the necessary fixes in the packages that depend on OpenGL to work with the restructuring. This also attempts to create a consistent variant for specifying glx or osmesa.
2022-05-24Add a command to generate a local mirror for bootstrapping (#28556)Massimiliano Culpo1-24/+5
This PR builds on #28392 by adding a convenience command to create a local mirror that can be used to bootstrap Spack. This is to overcome the inconvenience in setting up this mirror manually, which has been reported when trying to setup Spack on air-gapped systems. Using this PR the user can create a bootstrapping mirror, on a machine with internet access, by: % spack bootstrap mirror --binary-packages /opt/bootstrap ==> Adding "clingo-bootstrap@spack+python %apple-clang target=x86_64" and dependencies to the mirror at /opt/bootstrap/local-mirror ==> Adding "gnupg@2.3: %apple-clang target=x86_64" and dependencies to the mirror at /opt/bootstrap/local-mirror ==> Adding "patchelf@0.13.1:0.13.99 %apple-clang target=x86_64" and dependencies to the mirror at /opt/bootstrap/local-mirror ==> Adding binary packages from "https://github.com/alalazo/spack-bootstrap-mirrors/releases/download/v0.1-rc.2/bootstrap-buildcache.tar.gz" to the mirror at /opt/bootstrap/local-mirror To register the mirror on the platform where it's supposed to be used run the following command(s): % spack bootstrap add --trust local-sources /opt/bootstrap/metadata/sources % spack bootstrap add --trust local-binaries /opt/bootstrap/metadata/binaries The mirror has to be moved over to the air-gapped system, and registered using the commands shown at prompt. The command has options to: 1. Add pre-built binaries downloaded from Github (default is not to add them) 2. Add development dependencies for Spack (currently the Python packages needed to use spack style) * bootstrap: refactor bootstrap.yaml to move sources metadata out * bootstrap: allow adding/removing custom bootstrapping sources This operation can be performed from the command line since new subcommands have been added to `spack bootstrap` * Add --trust argument to spack bootstrap add * Add a command to generate a local mirror for bootstrapping * Add a unit test for mirror creation
2022-05-23Deprecate `spack:concretization` over `concretizer:unify` (#30038)Harmen Stoppels1-0/+6
* Introduce concretizer:unify option to replace spack:concretization * Deprecate concretization * Make spack:concretization overrule concretize:unify for now * Add environment update logic to move from spack:concretization to spack:concretizer:reuse * Migrate spack:concretization to spack:concretize:unify in all locations * For new environments make concretizer:unify explicit, so that defaults can be changed in 0.19
2022-05-18Add license dir to config (#30135)robgics1-0/+3
* Change license dir from hard-coded to a configurable item * Change config item to be a string not an array Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2022-05-13Reuse concretization by default (#30396)Massimiliano Culpo1-2/+2
* Enable reuse by default in Spack * Update documentation to match new default * Configure pipelines not to reuse software
2022-05-10Add a Lua build-system (#28854)Tom Scogland1-1/+2
Reworking lua to allow easier substitution of the base lua implementation. Also adding in a maintained version of luajit and re-factoring the entire stack to use a custom build-system to centralize functionality like environment variable management and luarocks installation. The `lua-lang` virtual is now versioned so that a package that requires Lua 5.1 semantics can get any lua, but one that requires 5.2 will only get upstream lua. The luaposix package requires lua-bit32, but only when built with a lua conforming to version 5.1. This adds the package, and the dependencies, but exposed a problem with luarocks dependency detection. Since we're installing each package in its own "tree" and there's no environment variable to list extra trees, spack now generates a luarocks config file that lists all the trees of all the dependencies, and references it by setting `LUAROCKS_CONFIG` in the build environment of every LuaPackage. This allows luarocks to find the spack installed dependencies correctly rather than trying (and failing) to download them. Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Tom Scogland <tscogland@llnl.gov> Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-04-25ASP-based solver: allow configuring target selection (#29835)Massimiliano Culpo1-0/+13
* ASP-based solver: allow configuring target selection This commit adds a new "concretizer:targets" configuration section, and two options under it. - "concretizer:targets:granularity" allows switching from considering only generic targets to consider all possible microarchitectures. - "concretizer:targets:host_compatible" instead controls whether we can concretize for microarchitectures that are incompatible with the current host. * Add documentation * Add unit-tests
2022-04-13Add support for Python 3.10 (#29581)Massimiliano Culpo1-4/+13
* Add support for Python 3.10 * Update unit-tests to use 3.10 * Update Getting started section of the docs * Update bootstrap action
2022-04-08Deprecate top-level module config (#28659)Harmen Stoppels1-6/+3
* Ignore top-level module config; add auto-update In Spack 0.17 we got module sets (modules:[name]:[prop]), and for backwards compat modules:[prop] was short for modules:default:[prop]. But this makes it awkward to define default config for the "default" module set. Since 0.17 is branched off, we can now deprecate top-level module config (that is, just ignore it with a warning). This PR does that, and it implements `spack config update modules` to make upgrading easy (we should have added that to 0.17 already...) It also removes references to `dotkit` stuff which was already deprecated in 0.13 and could have been removed in 0.14. Prefix inspections are the only exception, since the top-level prefix inspections used for `spack load` and `spack env activate`.
2022-03-17Expand external find for Windows (#27588)John Parent1-1/+1
* Incorporate new search location * Add external user option * proper doc string * Explicit commands in getting started * raise during chgrp on Win recover installer changes Notate admin privleges Windows phase install hooks Find external python and install ninja (#23496) Allow external find python to find windows python and spack install ninja Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17Packaging of netlib-lapack for windows (#24993)Jared Popelar1-0/+5
MSVC's internal CMake and Ninja now detected by spack external find and added to packages.yaml Saving progress on packaging zlib for Windows Fixing the shared CMake flag * Loading Intel's ifx Fortran compiler into MSVC; if there are multiple versions of MSVC installed and detected, ifx will only be placed into the first block written in compilers.yaml. The version number of ifx can be detected using MSVC's version flag (instead of /QV) by using ignore_version_errors. This commit also provides support for detection of Intel compilers in their own compiler block by adding ifx.exe to the fc/f77_name blocks inside intel.py * Giving CMake a Fortran compiler argument * Adding patch file for removing duplicated mangling header for versions 3.9.1 and older; static and shared now successfully building on Windows * Have netlib-lapack depend on ninja@1.10 Co-authored-by: John R. Cary <cary@txcorp.com> Co-authored-by: Jared Popelar <jpopelar@txcorp.com> Making a default config.yaml for Windows Small path length for build_stage Provide more prerequisite details, mention default config.yaml Killing an unnecessary setvars call Replacing some lost changes, proofreading, updating windows-supported package list Co-authored-by: John Parent <john.parent@kitware.com>
2022-02-16config: add a new `concretizer` config sectionTodd Gamblin2-4/+24
The concretizer is going to grow to have many more configuration, and we really need some structured config for that. * We have the `config:concretizer` option that chooses the solver, but extending that is awkward (we'd need to replace a string with a `dict`) and the solver choice will be deprecated eventually. * We have the `concretization` option in environments, but it's not a top-level config section -- it's just for environments, and it also only admits a string right now. To avoid overlapping with either of these and to allow the most extensibility in the future, this adds a new `concretizer` config section that can be used in and outside of environments. There is only one option right now: `reuse`. This can expand to include other options later. Likely, we will soon deprecate `config:concretizer` and warn when the user doesn't use `clingo`, and we will eventually (sometime later) move the `together` / `separately` options from `concretization` into the top-level `concretizer` section. This commit just adds the new section and schema. Fully wiring it up is TBD.
2022-01-18Switch lmod default all:autoload from none to direct (#28357)Harmen Stoppels1-0/+6
* Switch lmod module all autoload default from none to direct * Fix the docs
2022-01-11Packaging: Virtual package for libllvm (#27200)kwryankrattiger1-0/+1
Add an abstraction around libllvm to allow libllvm providers to be specified for all packages. This is targeting allowing mesa to build against llvm-amdgpu or intel-llvm or llvm or any other custom llvm variant that arises for specific GPU toolchains
2021-10-28config: fix `SPACK_DISABLE_LOCAL_CONFIG`, remove `$user_config_path` (#27022)Todd Gamblin1-2/+2
There were some loose ends left in ##26735 that cause errors when using `SPACK_DISABLE_LOCAL_CONFIG`. - [x] Fix hard-coded `~/.spack` references in `install_test.py` and `monitor.py` Also, if `SPACK_DISABLE_LOCAL_CONFIG` is used, there is the issue that `$user_config_path`, when used in configuration files, makes no sense, because there is no user config scope. Since we already have `$user_cache_path` in configuration files, and since there really shouldn't be *any* data stored in a configuration scope (which is what you'd configure in `config.yaml`/`bootstrap.yaml`/etc., this just removes `$user_config_path`. There will *always* be a `$user_cache_path`, as Spack needs to write files, but we shouldn't rely on the existence of a particular configuration scope in the Spack code, as scopes are configurable, both in number and location. - [x] Remove `$user_config_path` substitution. - [x] Fix reference to `$user_config_path` in `etc/spack/deaults/bootstrap.yaml` to refer to `$user_cache_path`, which is where it was intended to be.
2021-10-26config: overrides for caches and system and user scopes (#26735)Harmen Stoppels2-6/+6
Spack's `system` and `user` scopes provide ways for administrators and users to set global defaults for all Spack instances, but for use cases where one wants a clean Spack installation, these scopes can be undesirable. For example, users may want to opt out of global system configuration, or they may want to ignore their own home directory settings when running in a continuous integration environment. Spack also, by default, keeps various caches and user data in `~/.spack`, but users may want to override these locations. Spack provides three environment variables that allow you to override or opt out of configuration locations: * `SPACK_USER_CONFIG_PATH`: Override the path to use for the `user` (`~/.spack`) scope. * `SPACK_SYSTEM_CONFIG_PATH`: Override the path to use for the `system` (`/etc/spack`) scope. * `SPACK_DISABLE_LOCAL_CONFIG`: set this environment variable to completely disable *both* the system and user configuration directories. Spack will only consider its own defaults and `site` configuration locations. And one that allows you to move the default cache location: * `SPACK_USER_CACHE_PATH`: Override the default path to use for user data (misc_cache, tests, reports, etc.) With these settings, if you want to isolate Spack in a CI environment, you can do this: export SPACK_DISABLE_LOCAL_CONFIG=true export SPACK_USER_CACHE_PATH=/tmp/spack This is a stop-gap approach until we have figured out how to deal with the system and user config scopes more generally, as there are plans to potentially / eventually get rid of them. **User config** Spack is a bit of a pain when you have: - a shared $HOME folder across different systems. - multiple Spack versions on the same system. **System config** - On shared systems with a versioned programming environment / toolkit, system administrators want to provide config for each version (e.g. 21.09, 21.10) of the programming environment, and the user Spack instance should be able to pick this up without a steep learning curve. - On shared systems the user should be able to opt out of the hard-coded config scope in /etc/spack, since it may be incompatible with their particular instance. Currently Spack can only opt out of all config scopes through overrides with `"config:":`, `"packages:":`, but that also drops the defaults config, which would have to be repeated, which is undesirable, especially the lengthy packages.yaml. An example use case is: having config in this folder: ``` /path/to/programming/environment/{version}/{compilers,packages}.yaml ``` and have `module load spack-system-config` set the variable ``` SPACK_SYSTEM_CONFIG_PATH=/path/to/programming/environment/{version} ``` where the user no longer has to worry about what `{version}` they are on. **Continuous integration** Finally, there is the use case of continuous integration, which may clone an arbitrary Spack version, which optimally should not pick up system or user config from the previous run (like may happen in classical bare metal non-containerized filesystem side effect ridden jenkins pipelines). In fact this is very similar to how spack itself tries to avoid picking up system dependencies during builds... **But environments solve this?** - You could do `include`s in environment files to get similar behavior to the spack_system_config_path example, but environments require you to: 1) require paths to individual config files, not directories. 2) fail if the listed config file does not exist - They allow you to override config scopes, but this is generally too rigurous, as it requires you to repeat the default config, in particular packages.yaml, and just defies the point of layered config. Co-authored-by: Tom Scogland <tscogland@llnl.gov> Co-authored-by: Tim Fuller <tjfulle@sandia.gov> Co-authored-by: Steve Leak <sleak@lbl.gov> Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2021-10-11installer: Support showing status information in terminal title (#16259)Michael Kuhn1-1/+6
Installing packages with a lot of dependencies does not have an easy way of judging the current progress (apart from running `spack spec -I pkg` in another terminal). This change allows Spack to update the terminal's title with status information, including its current progress as well as information about the current and total number of packages.
2021-09-14Make clingo the default solver (#25502)Massimiliano Culpo1-4/+3
Modifications: - [x] Change `defaults/config.yaml` - [x] Add a fix for bootstrapping patchelf from sources if `compilers.yaml` is empty - [x] Make `SPACK_TEST_SOLVER=clingo` the default for unit-tests - [x] Fix package failures in the e4s pipeline Caveats: 1. CentOS 6 still uses the original concretizer as it can't connect to the buildcache due to issues with `ssl` (bootstrapping from sources requires a C++14 capable compiler) 1. I had to update the image tag for GitlabCI in e699f14. 1. libtool v2.4.2 has been deprecated and other packages received some update
2021-08-18Bootstrap clingo from binaries (#22720)Massimiliano Culpo1-0/+25
* Bootstrap clingo from binaries * Move information on clingo binaries to a JSON file * Add support to bootstrap on Cray Bootstrapping on Cray requires, at the moment, to swap the platform when looking for binaries - due to #22800. * Add SHA256 verification for bootstrapped software Use sha256 verification for binaries necessary to bootstrap the concretizer and gpg for signature verification * patchelf: use Spec._old_concretize() to bootstrap As noted in #24450 we may happen to need the concretizer when bootstrapping clingo. In that case only the old concretizer is available. * Add a schema for bootstrapping methods Two fields have been added to bootstrap.yaml: "sources" which lists the methods available for bootstrapping software "trusted" which records if a source is trusted or not A subcommand has been added to "spack bootstrap" to list the sources currently available. * Methods used for bootstrapping are configurable from bootstrap:sources The function that tries to ensure a given Python module is importable now tries bootstrapping methods in the same order as they are defined in `bootstrap.yaml` * Permit to trust/untrust bootstrapping methods * Add binary tests for MacOS, Ubuntu * Add documentation * Add a note on bash
2021-08-02document config option "url_fetch_method" (#24638)loulawrence1-0/+4
- Change config from the undocumented `use_curl: true/false` to `url_fetch_method: urllib/curl`. - Documentation of `url_fetch_method` in `defaults/config.yaml` - Default fetch option explicitly set to `urllib` for users who may not have curl on their system To upgrade from `use_curl` to `url_fetch_method`, run `spack config update config`
2021-07-27pbs: new virtual package (#24568)Seth R. Johnson1-0/+1
* pbs: new virtual package Some of our clusters have an older installation of libtorque and tm.h that are *not* from OpenPBS. Using the current openpbs dependency for openmpi causes concretization errors due to restrictions on older python and hwloc requirements that don't apply, even with an external non-buildable installation. The new 'torque' bundle package allows users to point to that external installation without problems. Detailed description of torque by Sergey Kosukhin <skosukhin@gmail.com>
2021-07-12Enable/disable bootstrapping and customize store location (#23677)Massimiliano Culpo1-0/+7
* Permit to enable/disable bootstrapping and customize store location This PR adds configuration handles to allow enabling and disabling bootstrapping, and to customize the store location. * Move bootstrap related configuration into its own YAML file * Add a bootstrap command to manage configuration
2021-06-28Hdf5 cmake (#18937)Larry Knox1-1/+1
* Switch hdf5 package from autotools to cmake. * Add variant for building with zlib, default to ON. * Update for format requirements. * Format change. * Fix breakage from last merge from develop. Switch szip to use libaec (unrestricted encryption). Remove 'static' variant: static libs will only be installed when ~shared. * Improve args based on suggestions from pull request. * Update code URL to github.com Add/modify 4 depends_on lines to fix running "spack graph --deptype=link hdf5". * Remove trailing whitespace. * Remove dependencies added solely to make "spack greph --type=link" work. * Add new version HDF5 1.8.22. * Remove unnecessary java_check. * Fix whitespace for style checks. * Reverted zlib version dependency to 1.1.2:. zlib variant removed. api version default renamed "default". * Remove blank line. * Whitespace corrections. * iRemoved unnecessary 'debug' variant. * Fix typo in version number in conflict for '+szip'. * Set default for tools variant to True. Remove patch functions dependent on 'libtool' file that cmake doesn't produce. * Remove line to set ONLY_SHARED_LIBS to true. Add post_install code to install only one version of tools with shared linkage and original tool names. * Remove trailing white space and import of glob package not used. * Leave BUILD_TESTING set to default which is ON. * Remove post_install code to install only one version of tools because some dependent packages running tests in e4s testing are using h5diff-shared. Keep both tools versions for now. * No longer need to import os.
2021-06-15add irep and lua-lang virtual dependency (#22492)Richarda Butler1-0/+1
This adds a package for `irep`, a tool for reading `lua` input decks from Fortran, C, and C++. `irep` can be built with either `lua` or `luajit`. To address this, we also add a virtual package for lua called `lua-lang`. `luajit` isn't, by default, a drop-in replacement for `lua`, but we add a `+lualinks` variant to it that adds symlinks that make it behave like `lua@5.1`. With this variant enabled, it provides the `lua-lang` virtual. `lua` always provides `lua-lang`. - [x] add `irep` package - [x] add `+lualinks` variant to `lua-luajit` - [x] create `lua-lang` virtual, provided by `lua` and `luajit+lualinks` Co-authored-by: Kayla Richarda Butler <butler59@quartz1148.llnl.gov> Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2021-06-11Fixing provides directive for intel-oneapi-onedal package (#24108)Nikolay Petrov1-0/+1
2021-06-04apply default linux prefix inspections to all module sets (#24151)Greg Becker1-6/+5
2021-05-28Separable module configuration -- without the bugs this time (#23703)Greg Becker3-17/+24
Currently, module configurations are inconsistent because modulefiles are generated with the configs for the active environment, but are shared among all environments (and spack outside any environment). This PR fixes that by allowing Spack environments (or other spack config scopes) to define additional sets of modules to generate. Each set of modules can enable either lmod or tcl modules, and contains all of the previously available module configuration. The user defines the name of each module set -- the set configured in Spack by default is named "default", and is the one returned by module manipulation commands in the absence of user intervention. As part of this change, the module roots configuration moved from the config section to inside each module configuration. Additionally, it adds a feature that the modulefiles for an environment can be configured to be relative to an environment view rather than the underlying prefix. This will not be enabled by default, as it should only be enabled within an environment and for non-default views constructed with separate projections per-spec.
2021-05-28Use AWS CloudFront for source mirror (#23978)Todd Gamblin1-1/+1
Spack's source mirror was previously in a plain old S3 bucket. That will still work, but we can do better. This switches to AWS's CloudFront CDN for hosting the mirror. CloudFront is 16x faster (or more) than the old bucket. - [x] change mirror to https://mirror.spack.io
2021-05-27Add fuse virtual dependency, new macfuse package (#23904)Adam J. Stewart2-6/+5
2021-05-26defaults/cray: use modules.yaml from defaults/linux (#23932)eugeneswalker1-0/+21
2021-05-25Add xxd for hsa-rocr-dev build script (#23855)Harmen Stoppels1-4/+5
2021-05-17Revert "Separable module configurations (#22588)" (#23674)Harmen Stoppels4-27/+22
This reverts commit cefbe48c89209dc3df654795644973b1885cdea4.
2021-05-14Separable module configurations (#22588)Greg Becker4-22/+27
Currently, module configurations are inconsistent because modulefiles are generated with the configs for the active environment, but are shared among all environments (and spack outside any environment). This PR fixes that by allowing Spack environments (or other spack config scopes) to define additional sets of modules to generate. Each set of modules can enable either lmod or tcl modules, and contains all of the previously available module configuration. The user defines the name of each module set -- the set configured in Spack by default is named "default", and is the one returned by module manipulation commands in the absence of user intervention. As part of this change, the module roots configuration moved from the `config` section to inside each module configuration. Additionally, it adds a feature that the modulefiles for an environment can be configured to be relative to an environment view rather than the underlying prefix. This will not be enabled by default, as it should only be enabled within an environment and for non-default views constructed with separate projections per-spec. TODO: - [x] code changes to support multiple module sets - [x] code changes to support modules relative to a view - [x] Tests for multiple module configurations - [x] Tests for modules relative to a view - [x] Backwards compatibility for module roots from config section - [x] Backwards compatibility for default module set without the name specified - [x] Tests for backwards compatibility
2021-03-30Make -j flag less exceptional (#22360)Harmen Stoppels1-5/+7
* Make -j flag less exceptional The -j flag in spack behaves differently from make, ctest, ninja, etc, because it caps the number of jobs to an arbitrary number 16. Spack will behave like other tools if `spack install` uses a reasonable default, and `spack install -j <num>` *overrides* that default. This will be particularly useful for Spack usage outside of a traditional HPC context and for HPC centers that encourage users to compile on login nodes with many cores instead of on compute nodes, which has become increasingly common as individual nodes have more cores. This maintains the existing default value of min(num_cpus, 16). However, as it is right now, Spack does a poor job at determining the number of cpus on linux, since it doesn't take cgroups into account. This is particularly problematic when using distributed builds with slurm. This PR also introduces `spack.util.cpus.cpus_available()` to consolidate knowledge on determining the number of available cores, and improves core detection for linux. This should also improve core detection for Docker/ Kubernetes, which also use cgroups.
2021-03-03zig: add new package at v0.7.1 (#22046)Massimiliano Culpo1-0/+1
2021-02-23Drop compiler variables from spack load (#21699)Harmen Stoppels1-10/+0
Drops: * C_INCLUDE_PATH * CPLUS_INCLUDE_PATH * LIBRARY_PATH * INCLUDE We already decided to use C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, INCLUDE over CPATH here: https://github.com/spack/spack/pull/14749 However, none of these flags apply to Fortran on Linux. So for consistency it seems better to make the user use -I and -L flags by hand or through pkgconfig.
2021-02-09Procedure to deprecate old versions of software (#20767)Adam J. Stewart1-1/+6
* Procedure to deprecate old versions of software * Add documentation * Fix bug in logic * Update tab completion * Deprecate legacy packages * Deprecate old mxnet as well * More explicit docs
2021-02-04config: (darwin only) change prefix of external libuuid (#21480)Laurent Aphecetche1-1/+1