summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2021-05-28aocc version detection (#23907)Greg Becker1-1/+4
2021-05-27bugfix: mirror index shows missing packages (#23939)Greg Becker3-4/+66
- [x] add `in_buildcache` field to DB records to indicate what parts of an index, which includes roots and dependencies, are in the buildcache. - [x] add `mark()` method to DB for setting values on single nodes of the DAG.
2021-05-27only readlink on links (#23948)Greg Becker1-1/+1
2021-05-26allow whitespace formatting in variant descriptions (#23853)Greg Becker1-4/+6
2021-05-25adding json export for spack blame (#23417)Vanessasaurus3-14/+121
I would like to be able to export (and save and then load programatically) spack blame metadata, so this commit adds a spack blame --json argument, along with developer docs for it Signed-off-by: vsoch <vsoch@users.noreply.github.com> Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2021-05-25first set of work to allow for saving local results with spack monitor (#23804)Vanessasaurus5-22/+154
This work will come in two phases. The first here is to allow saving of a local result with spack monitor, and the second will add a spack monitor command so the user can do spack monitor upload. Signed-off-by: vsoch <vsoch@users.noreply.github.com> Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2021-05-25Fix cross references in inteloneapipackage doc (#23744)Robert Cohn1-7/+7
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-05-25Stand-alone/Smoke tests: copy cached test sources to test stage (#23713)Tamara Dahlgren8-26/+104
2021-05-25MesonPackage: make "default_library" a multi-valued variant (#23540)Harmen Stoppels1-6/+7
Currently if one package does `depends_on('pkg default_library=shared')` and another does `depends_on('pkg default_library=both')`, you'd get a concretization error. With this PR one package can do `depends_on('pkg default_library=shared')` and another depends_on('default_library=static'), and it would concretize to `pkg default_library=shared,static` Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-05-25Fix packaging guide table's build system links (#23879)Tamara Dahlgren1-9/+9
2021-05-25Fix hyperlink formatting in docs (#23846)HDF-EOS Tools Information Center1-1/+1
2021-05-22Fix makefile filter suggestions (#23856)Seth R. Johnson2-6/+8
Bash has a builtin `fc` that will override the compiler if you use "fc", so it's better to use the full spack-supplied compiler path. Additionally, the filter regex in the docs was wrong: it replaced the entire assignment operation with the RHS.
2021-05-21Modification to R environment (#23623)Glenn Johnson1-2/+1
* Modification to R environment This PR modifies how the R environmnet is presented, and fixes installing the standalone Rmath library. - The Rmath build and install methods are combined into one - Set parallel=False when installing Rmath - remove the run environment that set up variables for libraries and headers that are not really needed, and pollute the environment. * Add setup_run_environment back - Add back the setup_run_environment with LD_LIBRARY_PATH and PKG_CONFIG_PATH. - Adjust documentation to reflect the current code.
2021-05-20spack: update archspecMassimiliano Culpo3-7/+172
This fixes the detection of Apple M1 and adds virtual levels for x86_64 architectures
2021-05-20versions: do not drop 2.0.X if 2.0 is a declared version in the package (#23217)Valentin Volkl2-6/+30
2021-05-20Cray: fix extracting paths from module files (#23472)Harmen Stoppels2-3/+13
Co-authored-by: Tiziano Müller <tm@dev-zero.ch>
2021-05-19Write junit-report to reports directory to allow installation from ↵Valentin Volkl2-1/+3
read-only spack (#20158)
2021-05-19adding support to tag a buildvsoch3-5/+26
This will be useful to run multiple build experiments and organize by name Signed-off-by: vsoch <vsoch@users.noreply.github.com>
2021-05-19Add caret/hat to spack spec help documentation (#23758)Jamie Finney1-0/+1
Co-authored-by: Jamie Finney <finneyjm@ornl.gov>
2021-05-19monitor: fix issue with attribute lookup (#23773)Vanessasaurus1-1/+1
Signed-off-by: vsoch <vsoch@users.noreply.github.com> Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2021-05-18docs/packaging guide: Reference test stage directory (#23707)Tamara Dahlgren1-3/+15
2021-05-18Bugfix: fetching oddly-named resources from local mirrors (#23300)Michael Kuhn1-1/+9
Spack uses curl to fetch URL resources. For locally-stored resources it uses curl's file protocol; when using this protocol, curl expects that the URL encoding conforms to RFC 3986 (which reserves characters like '?' and '=' for special use). We were not performing this encoding, and found a resource where curl was interpreting this in an unfavorable way (succeeding, but producing an empty file). This commit properly encodes URLs when using curl's file protocol. This error did not likely come up before because in most contexts Spack was either fetching via http or it was using URLs without offending characters (for example, the sha-based URLs in mirrors never contain these characters).
2021-05-17Revert "Separable module configurations (#22588)" (#23674)Harmen Stoppels29-491/+183
This reverts commit cefbe48c89209dc3df654795644973b1885cdea4.
2021-05-17performance: speed up existence checks in packages (#23661)Todd Gamblin3-24/+37
Spack doesn't require users to manually index their repos; it reindexes the indexes automatically when things change. To determine when to do this, it has to `stat()` all package files in each repository to make sure that indexes up to date with packages. We currently index virtual providers, patches by sha256, and tags on packages. When this was originally implemented, we ran the checker all the time, at startup, but that was slow (see #7587). But we didn't go far enough -- it still consults the checker and does all the stat operations just to see if a package exists (`Repo.exists()`). That might've been a wash in 2018, but as the number of packages has grown, it's gotten slower -- checking 5k packages is expensive and users see this for small operations. It's a win now to make `Repo.exists()` check files directly. **Fix:** This PR does a number of things to speed up `spack load`, `spack info`, and other commands: - [x] Make `Repo.exists()` check files directly again with `os.path.exists()` (this is the big one) - [x] Refactor `Spec.satisfies()` so that a checking for virtual packages only happens if needed (avoids some calls to exists()) - [x] Avoid calling `Repo.exists(spec)` in `Repo.get()`. `Repo.get()` will ultimately try to load a `package.py` file anyway; we can let the failure to load it indicate that the package doesn't exist, and avoid another call to exists(). - [x] Fix up some comments in spec parsing - [x] Call `UnknownPackageError` more consistently in `repo.py`
2021-05-15some fixes for command help strings (#23658)Todd Gamblin3-8/+5
- [x] `analyze` isn't commonly used; move it to long help (`spack -H` vs `spack -h`). Give it its own section. - [x] make it clear from `spack -h` that `spack module` can generate module files - [x] shorten help for `spack style`
2021-05-15do not sort projections alphabetically (#23649)Greg Becker1-3/+7
* do not sort projections alphabetically * add assertion for ordered dict
2021-05-14Separable module configurations (#22588)Greg Becker29-183/+491
Currently, module configurations are inconsistent because modulefiles are generated with the configs for the active environment, but are shared among all environments (and spack outside any environment). This PR fixes that by allowing Spack environments (or other spack config scopes) to define additional sets of modules to generate. Each set of modules can enable either lmod or tcl modules, and contains all of the previously available module configuration. The user defines the name of each module set -- the set configured in Spack by default is named "default", and is the one returned by module manipulation commands in the absence of user intervention. As part of this change, the module roots configuration moved from the `config` section to inside each module configuration. Additionally, it adds a feature that the modulefiles for an environment can be configured to be relative to an environment view rather than the underlying prefix. This will not be enabled by default, as it should only be enabled within an environment and for non-default views constructed with separate projections per-spec. TODO: - [x] code changes to support multiple module sets - [x] code changes to support modules relative to a view - [x] Tests for multiple module configurations - [x] Tests for modules relative to a view - [x] Backwards compatibility for module roots from config section - [x] Backwards compatibility for default module set without the name specified - [x] Tests for backwards compatibility
2021-05-13spec: simplify __str__ implementation (#23593)Massimiliano Culpo2-13/+10
The implementation for __str__ has been simplified to traverse the spec directly, and doesn't call anymore the flat_dependencies method. Dead code has been removed.
2021-05-13cc: change mode to ccld for loopopt edit (#23482)Frank Willmore2-4/+13
For configure (e.g. for hdf5) to pass, this option needs to be pulled out when invoked in ccld mode. I thought it had fixed the issue but I still saw it after that. After some digging, my guess is that I was able to get hdf5 to build with ifort instead of ifx. Lot of overlapping changes occurring at the time, as it were. There are still outstanding issues building hdf5 with ifx, and Intel is looking into what appears to be a compiler bug, but this manifests during build and is likely a separate issue. I have verified that the making the edit in 'ccld' mode removes the -loopopt=0 and enables hdf5 to pass configure. It should be fine to make the edit in 'ld' mode as well, but I have not tested that and didn't include an -or- condition for it.
2021-05-13config key error: fix format string (#23610)Greg Becker1-1/+1
2021-05-13env views: make view updates atomic (#23476)Greg Becker6-72/+172
Currently, environment views blink out of existence during the view regeneration, and are slowly built back up to their new and improved state. This is not good if other processes attempt to access the view -- they can see it in an inconsistent state. This PR fixes makes environment view updates atomic. This requires a level of indirection (via symlink, similar to nix or guix) from the view root to the underlying implementation on the filesystem. Now, an environment view at `/path/to/foo` is a symlink to `/path/to/._foo/<hash>`, where `<hash>` is a hash of the contents of the view. We construct the view in its content-keyed hash directory, create a new symlink to this directory, and atomically replace the symlink with one to the new view. This PR has a couple of other benefits: * It future-proofs environment views so that we can implement rollback. * It ensures that we don't leave users in an inconsistent state if building a new view fails for some reason. For background: * there is no atomic operation in posix that allows for a non-empty directory to be replaced. * There is an atomic `renameat2` in the linux kernel starting in version 3.15, but many filesystems don't support the system call, including NFS3 and NFS4, which makes it a poor implementation choice for an HPC tool, so we use the symlink approach that others tools like nix and guix have used successfully.
2021-05-12ASP-based solver: account for deprecated versions (#23491)Massimiliano Culpo4-1/+43
fixes #22351 The ASP-based solver now accounts for the presence in the DAG of deprecated versions and tries to minimize their number at highest priority.
2021-05-11Environments: add run deps to shell modifications (#23485)Peter Scheibel2-12/+42
When adding an Environment to a user's shell, Spack was only adding root specs. This now includes run dependencies of root specs.
2021-05-11ASP-based solver: variants set from cli are considered as defaults (#23542)Massimiliano Culpo3-7/+40
Variants explicitly set in an abstract root spec are considered as defaults for the package they refer to, and they override what is in packages.yaml and in package.py. This is relevant only for multi-valued variants, where a constraint may extend an already default value.
2021-05-11cray: fix parsing of module list (#23566)Howard Pritchard1-0/+1
The code for guessing cpu archtype based on craype modules names got confused, at least on LLNL RZ prototype systems. In particular a (L) or (D) at the end of a craype-x86-xxx or other cpu architecture module was geting the logic confused. With this patch, any white space + remaining characters in the moduel name are removed. Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2021-05-11Updates and format tweaks to the release documentation (#22053)Tamara Dahlgren1-93/+131
2021-05-11Documentation: Refinement of "Checking an installation" (#22210)Tamara Dahlgren3-164/+703
There have been a lot of questions and some confusion recently surrounding Spack installation test capabilities so this PR is intended to clean up and refine the documentation for "Checking an installation". It aims to better distinguish between checks that are performed during an installation (i.e., build-time tests) and those that can be done days and weeks after the software has been installed (i.e., install (or smoke) tests).
2021-05-10bugfix: correct force_autoreconf method syntax (#23546)Tamara Dahlgren1-1/+1
2021-05-10compare full old_prefix and new_prefix instead of layout_root (#23506)eugeneswalker1-1/+1
2021-05-08clingo: don't skip tests that deal with file permissionsMassimiliano Culpo3-16/+0
When we first merged the ASP-based solver, unit-tests were run in a Docker container with root permissions and that was preventing a few tests to succeed. Since some time though, clingo is tested as a regular user within Github Actions VMs, so we should start to run checks again.
2021-05-07install cmd: --no-add in an env installs w/out concretize and addScott Wittenburg4-8/+196
In an active concretize environment, support installing one or more cli specs only if they are already present in the environment. The `--no-add` option is the default for root specs, but optional for dependency specs. I.e. if you `spack install <depspec>` in an environment, the dependency-only spec `depspec` will be added as a root of the environment before being installed. In addition, `spack install --no-add <spec>` fails if it does not find an unambiguous match for `spec`.
2021-05-07return concrete spec (as is advertised in the docstring)Peter Josef Scheibel1-1/+1
2021-05-07fix check when retrieving matching spec from environment when there is a ↵Peter Josef Scheibel2-1/+19
match with a root spec as well as with a dependency of a root spec
2021-05-07compilers: aocc is now also available as a Cray PrgEnv (#23289)Tiziano Müller1-0/+3
2021-05-07Show useful error in build-env (#23458)Harmen Stoppels2-11/+21
Instead of an out of bounds error tell the user to provide a spec
2021-05-06ASP-based solver: minimize mismatch of targets (#23462)Massimiliano Culpo1-32/+19
Like compilers targets now try to minimize mismatches, instead of maximizing matches. Deduction of mismatches is reworked to be the opposite of a match, since computing that is faster.
2021-05-06ASP-based solver: no intermediate package for concretizing together (#23307)Massimiliano Culpo2-0/+33
The ASP-based solver can natively manage cases where more than one root spec is given, and is able to concretize all the roots together (ensuring one spec per package at most). Modifications: - [x] When concretising together an environment the ASP-based solver calls directly its `solve` method rather than constructing a temporary fake root package.
2021-05-06Reduce visual noise during distributed build (#23338)Harmen Stoppels2-3/+7
2021-05-06Put a module object in sys.modules before executing module code (#23269)Massimiliano Culpo2-2/+39
The loading protocol mandates that the the module we are going to import needs to be already in sys.modules before its code is executed, so to prevent unbounded recursions and multiple loading. Loading a module from file exits early if the module is already in sys.modules
2021-05-05intel-oneapi packages: support root installs (#23401)Robert Cohn1-0/+21
When installing OneAPI packages as root (e.g. in a container), the installer places cache files in /var/intel/installercache that interfere with future Spack installs. This ensures that when running an installation as a root user that this is removed.