summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2021-02-18Pipelines: Move PR testing stacks (currently only E4S) into spack (#21714)Scott Wittenburg3-37/+61
2021-02-18Fix template for Rpackage in spack create command (#21776)Tom Payerle1-1/+1
The signature for configure_args in the template for new RPackage packages was incorrect (different than what is defined and used in lib/spack/spack/build_systems/r.py) See issue #21774
2021-02-18Testing: use spack.store.use_store everywhere (#21656)Massimiliano Culpo10-175/+176
Keep spack.store.store and spack.store.db consistent in unit tests * Remove calls to monkeypatch for spack.store.store and spack.store.db: tests that used these called one or the other, which lead to inconsistencies (the tests passed regardless but were fragile as a result) * Fixtures making use of monkeypatch with mock_store now use the updated use_store function, which sets store.store and store.db consistently * subprocess_context.TestState now transfers the serializes and restores spack.store.store (without the monkeypatch changes this would have created inconsistencies)
2021-02-18Documentation fix: build_system configure_args for #21760 (#21761)Tom Payerle1-2/+2
Corrects the signature for configure_args (and therefore configure_vars) in documentation on RPackage build system to match the code See issue #21760
2021-02-18bugfix: relax racy test in fg/bg output (#21755)Todd Gamblin1-12/+6
Since signals are fundamentally racy, We can't bound the amount of time that the `test_foreground_background_output` test will take to get to 'on', we can only observe that it transitions to 'on'. So instead of using an arbitrary limit, just adjust the test to allow either 'on' or 'off' followed by 'on'. This should eliminate the spurious errors we see in CI.
2021-02-18Avoid spurious warning from clingo (#21731)Massimiliano Culpo1-0/+1
There's a spurious warning that occurs whenever a spec being concretized does not depend on a virtual provider under any possible configuration.
2021-02-17apple-clang: add correct path to compiler wrappers (#21662)Adam J. Stewart1-1/+1
Follow-up to #17110 ### Before ```bash CC=/Users/Adam/spack/lib/spack/env/clang/clang; export CC SPACK_CC=/usr/bin/clang; export SPACK_CC PATH=...:/Users/Adam/spack/lib/spack/env/apple-clang:/Users/Adam/spack/lib/spack/env/case-insensitive:/Users/Adam/spack/lib/spack/env:...; export PATH ``` ### After ```bash CC=/Users/Adam/spack/lib/spack/env/clang/clang; export CC SPACK_CC=/usr/bin/clang; export SPACK_CC PATH=...:/Users/Adam/spack/lib/spack/env/clang:/Users/Adam/spack/lib/spack/env/case-insensitive:/Users/Adam/spack/lib/spack/env:...; export PATH ``` `CC` and `SPACK_CC` were being set correctly, but `PATH` was using the name of the compiler `apple-clang` instead of `clang`. For most packages, since `CC` was set correctly, nothing broke. But for packages using `Makefiles` that set `CC` based on `which clang`, it was using the system compilers instead of the compiler wrappers. Discovered when working on `py-xgboost@0.90`. An alternative fix would be to copy the symlinks in `env/clang` to `env/apple-clang`. Let me know if you think there's a better way to do this, or to test this.
2021-02-16Delete unused "mutable_mock_store" fixture (#21632)Massimiliano Culpo1-31/+0
The fixture was introduced in #19690 maybe accidentally. It's not used in unit tests, and though it should be mutable it seems an exact copy of it's immutable version.
2021-02-16Pipelines: Temporary buildcache storage (#21474)Scott Wittenburg9-83/+427
Before this change, in pipeline environments where runners do not have access to persistent shared file-system storage, the only way to pass buildcaches to dependents in later stages was by using the "enable-artifacts-buildcache" flag in the gitlab-ci section of the spack.yaml. This change supports a second mechanism, named "temporary-storage-url-prefix", which can be provided instead of the "enable-artifacts-buildcache" feature, but the two cannot be used at the same time. If this prefix is provided (only "file://" and "s3://" urls are supported), the gitlab "CI_PIPELINE_ID" will be appended to it to create a url for a mirror where pipeline jobs will write buildcache entries for use by jobs in subsequent stages. If this prefix is provided, a cleanup job will be generated to run after all the rebuild jobs have finished that will delete the contents of the temporary mirror. To support this behavior a new mirror sub-command has been added: "spack mirror destroy" which can take either a mirror name or url. This change also fixes a bug in generation of "needs" list for each job. Each jobs "needs" list is supposed to only contain direct dependencies for scheduling purposes, unless "enable-artifacts-buildcache" is specified. Only in that case are the needs lists supposed to contain all transitive dependencies. This changes fixes a bug that caused the needs lists to always contain all transitive dependencies, regardless of whether or not "enable-artifacts-buildcache" was specified.
2021-02-16NAG compiler: disable forcing RPATH/RUNPATH (#21701)Sergey Kosukhin1-0/+10
2021-02-16Pipelines: DAG Pruning (#20435)Scott Wittenburg9-257/+827
Pipelines: DAG pruning During the pipeline generation staging process we check each spec against all configured mirrors to determine whether it is up to date on any of the mirrors. By default, and with the --prune-dag argument to "spack ci generate", any spec already up to date on at least one remote mirror is omitted from the generated pipeline. To generate jobs for up to date specs instead of omitting them, use the --no-prune-dag argument. To speed up the pipeline generation process, pass the --check-index-only argument. This will cause spack to check only remote buildcache indices and avoid directly fetching any spec.yaml files from mirrors. The drawback is that if the remote buildcache index is out of date, spec rebuild jobs may be scheduled unnecessarily. This change removes the final-stage-rebuild-index block from gitlab-ci section of spack.yaml. Now rebuilding the buildcache index of the mirror specified in the spack.yaml is the default, unless "rebuild-index: False" is set. Spack assigns the generated rebuild-index job runner attributes from an optional new "service-job-attributes" block, which is also used as the source of runner attributes for another generated non-build job, a no-op job, which spack generates to avoid gitlab errors when DAG pruning results in empty pipelines.
2021-02-13Correct the reference to the staged examples files (#21557)Tamara Dahlgren1-3/+3
2021-02-12Introduce a SPACK_PYTHON environment variable (#21222)Chuck Atkins1-0/+8
The SPACK_PYTHON environment variable can be set to a python interpreter to be used by the spack command. This allows the spack command itself to use a consistent and separate interpreter from whatever python might be used for package building.
2021-02-11unit test: refactored bindist.py (#21497)Massimiliano Culpo1-353/+156
Modifications: - Make use of SpackCommand objects wherever possible - Deduplicated code when possible - Moved cleaning of mirrors to fixtures - Ensure mock configuration has a clear initialization order
2021-02-11database: don't create a date object in `query()` unless it's necessaryTodd Gamblin1-5/+6
`query()` calls `datetime.datetime.fromtimestamp` regardless of whether a date query is being done. Guard this with an if statement to avoid the unnecessary work.
2021-02-11version: precompile regexes in Version constructorTodd Gamblin1-6/+9
2021-02-11spack_json: _strify should be a no-op in Python 3Todd Gamblin1-3/+7
_strify traverses *all* of the read-in data and converts strings, but only for Python 2. Ignore it in Python 3.
2021-02-11database: do not redundantly mark specs concreteTodd Gamblin2-5/+9
Speed up reads by not traversing each spec recursively when marking them concrete.
2021-02-11spec performance: don't parse in from_node_dictTodd Gamblin1-1/+3
Constructing a spec from a name instead of setting name directly forces from_node_dict to call Spec.parse(), which is slow. Avoid this by using a zero-arg constructor and setting name directly.
2021-02-10Added a context manager to swap architecturesMassimiliano Culpo5-34/+88
This solves a few FIXMEs in conftest.py, where we were manipulating globals and seeing side effects prior to registering fixtures. This commit solves the FIXMEs, but introduces a performance regression on tests that may need to be investigated
2021-02-10Move context manager to swap the current configuration into spack.configMassimiliano Culpo3-64/+78
The context manager can be used to swap the current configuration temporarily, for any use case that may need it.
2021-02-10Move context manager to swap the current store into spack.storeMassimiliano Culpo3-15/+32
The context manager can be used to swap the current store temporarily, for any use case that may need it.
2021-02-10repo: generalize "swap" context manager to also accept pathsMassimiliano Culpo16-61/+71
The method is now called "use_repositories" and makes it clear in the docstring that it accepts as arguments either Repo objects or paths. Since there was some duplication between this contextmanager and "use_repo" in the testing framework, remove the latter and use spack.repo.use_repositories across the entire code base. Make a few adjustment to MockPackageMultiRepo, since it was stating in the docstring that it was supposed to mock spack.repo.Repo and was instead mocking spack.repo.RepoPath.
2021-02-10Fix: Add a space between -isystem and the directory (#21599)Scott McMillan2-15/+15
Some compilers, such as the NV compilers, do not recognize -isystem dir when specified without a space. Works: -isystem ../include Does not work: -isystem../include This PR updates the compiler wrapper to include the space with -isystem.
2021-02-10Bugfix: environments/views on separate mounts (#20720)Greg Becker1-4/+13
Environment views fail when the tmpdir used for view generation is on a separate mount from the install_tree because the files cannot by symlinked between the two. The fix is to use an alternative tmpdir located alongside the view.
2021-02-09Procedure to deprecate old versions of software (#20767)Adam J. Stewart12-11/+124
* Procedure to deprecate old versions of software * Add documentation * Fix bug in logic * Update tab completion * Deprecate legacy packages * Deprecate old mxnet as well * More explicit docs
2021-02-09Changed stages permissions to 750 (#21425)BenWeber421-3/+3
2021-02-09Fixed uninstall rm parent folder race condition (#21424)BenWeber421-3/+12
2021-02-09Fix typo (#21567)Mark Meredith1-1/+1
2021-02-08Restore verbosity of patching (#21233)Adam J. Stewart1-6/+6
2021-02-05let intel consider cray-mpich as mpich (#21512)albestro1-0/+1
2021-02-04Fix usage of PythonPackage.test outside of PythonPackage (#20555)Adam J. Stewart1-4/+50
2021-02-04Python: add maintainer(s) (#21125)Adam J. Stewart1-0/+2
* Python: add maintainer(s) * Fix unit tests
2021-02-04spack external find: allow to search by tags (#21407)Massimiliano Culpo5-9/+55
This commit adds an option to the `external find` command that allows it to search by tags. In this way group of executables with common purposes can be grouped under a single name and a simple command can be used to detect all of them. As an example introduce the 'build-tools' tag to search for common development tools on a system
2021-02-03concretizer: simplify "fact" method (#21148)Massimiliano Culpo1-23/+5
The "fact" method before was dealing with multiple facts registered per call, which was used when we were emitting grounded rules from knowledge of the problem instance. Now that the encoding is changed we can simplify the method to deal only with a single fact per call.
2021-02-02add intel oneapi to compiler/pkg translations (#21448)Greg Becker1-1/+2
2021-02-01PythonPackage: fewer phases (#20738)Adam J. Stewart2-109/+1
2021-02-01R: add maintainer(s) (#21119)Adam J. Stewart1-0/+2
* R: add maintainer(s) * Add maintainer to base class
2021-02-01Python 3.10 support: collections.abc (#20441)Adam J. Stewart19-42/+140
2021-01-29Print groups properly for spack find -d (#20028)Yang Zongze1-1/+1
2021-01-28Make git fetching even quieter (for git v1.7.2+)(#21013)Adam J. Stewart1-0/+6
2021-01-27Add a context wrapper for mtime preservation (#21258)Sergey Kosukhin2-1/+44
Sometimes we need to patch a file that is a dependency for some other automatically generated file that comes in a release tarball. As a result, make tries to regenerate the dependent file using additional tools (e.g. help2man), which would not be needed otherwise. In some cases, it's preferable to avoid that (e.g. see #21255). A way to do that is to save the modification timestamps before patching and restoring them afterwards. This PR introduces a context wrapper that does that.
2021-01-27Doc default behavior of install tests (#21309)Mark C. Miller1-2/+5
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-01-27spack setup: remove the command for v0.17.0 (#20277)Adam J. Stewart3-204/+0
spack setup was deprecated in 0.16 and will be removed in 0.17 Follow-up to #18240
2021-01-22use module and package flags to get more correct mypy behavior (#21225)Tom Scogland2-32/+25
The first of my two upstream patches to mypy landed in the 0.800 tag that was released this morning, which lets us use module and package parameters with a .mypy.ini file that has a files key. This uses those parameters to check all of spack in style, but leaves the packages out for now since they are still very, very broken. If no package has been modified, the packages are not checked, but if one has they are. Includes some fixes for the log tests since they were not type checking. Should also fix all failures related to "duplicate module named package" errors. Hopefully the next drop of mypy will include my other patch so we can just specify the modules and packages in the config file to begin with, but for now we'll have to live with a bare mypy doing a check of the libs but not the packages. * use module and package flags to check packages properly * stop checking package files, use package flag for libs The packages are not type checkable yet, need to finish out another PR before they can be. The previous commit also didn't check the libraries properly, this one does.
2021-01-22Added @property stdcxx_libs to return -lstdc++ for AOCC compiler (#21145)AMD Toolchain Support2-0/+6
2021-01-21docs: Update the CudaPackage (build system) description (#20742)Tamara Dahlgren1-16/+101
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
2021-01-21Added ROCmPackage (build system) documentation (#20743)Tamara Dahlgren3-2/+127
2021-01-20store sbang_install_path in buildinfo, use for subsequent relocation (#20768)eugeneswalker1-0/+11
2021-01-20[WIP] relocate.py: parallelize test replacement logic (#19690)Nathan Hanford9-210/+281
* sbang pushed back to callers; star moved to util.lang * updated unit test * sbang test moved; local tests pass Co-authored-by: Nathan Hanford <hanford1@llnl.gov>