summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2019-12-23performance: add read transactions for `install_all()` and `install()`Todd Gamblin1-31/+35
Environments need to read the DB a lot when installing all specs. - [x] Put a read transaction around `install_all()` and `install()` to avoid repeated locking
2019-12-23lock transactions: avoid redundant reading in write transactionsTodd Gamblin2-1/+62
Our `LockTransaction` class was reading overly aggressively. In cases like this: ``` 1 with spack.store.db.read_transaction(): 2 with spack.store.db.write_transaction(): 3 ... ``` The `ReadTransaction` on line 1 would read in the DB, but the WriteTransaction on line 2 would read in the DB *again*, even though we had a read lock the whole time. `WriteTransaction`s were only considering nested writes to decide when to read, but they didn't know when we already had a read lock. - [x] `Lock.acquire_write()` return `False` in cases where we already had a read lock.
2019-12-23lock transactions: ensure that nested write transactions writeTodd Gamblin2-1/+64
If a write transaction was nested inside a read transaction, it would not write properly on release, e.g., in a sequence like this, inside our `LockTransaction` class: ``` 1 with spack.store.db.read_transaction(): 2 with spack.store.db.write_transaction(): 3 ... 4 with spack.store.db.read_transaction(): ... ``` The WriteTransaction on line 2 had no way of knowing that its `__exit__()` call was the last *write* in the nesting, and it would skip calling its write function. The `__exit__()` call of the `ReadTransaction` on line 1 wouldn't know how to write, and the file would never be written. The DB would be correct in memory, but the `ReadTransaction` on line 4 would re-read the whole DB assuming that other processes may have modified it. Since the DB was never written, we got stale data. - [x] Make `Lock.release_write()` return `True` whenever we release the *last write* in a nest.
2019-12-23lock transactions: fix non-transactional writesTodd Gamblin4-172/+285
Lock transactions were actually writing *after* the lock was released. The code was looking at the result of `release_write()` before writing, then writing based on whether the lock was released. This is pretty obviously wrong. - [x] Refactor `Lock` so that a release function can be passed to the `Lock` and called *only* when a lock is really released. - [x] Refactor `LockTransaction` classes to use the release function instead of checking the return value of `release_read()` / `release_write()`
2019-12-23performance: avoid repeated DB locking on view generationTodd Gamblin1-10/+13
`ViewDescriptor.regenerate()` checks repeatedly whether packages are installed and also does a lot of DB queries. Put a read transaction around the whole thing to avoid repeatedly locking and unlocking the DB.
2019-12-23added master branch version to mpip (#14284)Gregory Lee1-0/+2
2019-12-23gromacs: add v2019.5 (#14285)Christoph Junghans1-0/+1
2019-12-23Added conflict with %pgi in libpciaccess package.py with relevant links in ↵Nicholas Sly1-0/+9
comments. (#14281)
2019-12-23netlib-scalapack: Add int overflow patch (#14276)Piotr Luszczek2-0/+315
2019-12-22Add py-boto3 1.10.44 (#14271)Adam J. Stewart1-1/+3
2019-12-22add new package : libdap4 (#14267)Timo Heister1-0/+40
2019-12-22Add py-numpy 1.18.0 (#14269)Adam J. Stewart1-2/+6
2019-12-22Add awscli 1.16.308 (#14270)Adam J. Stewart1-10/+10
2019-12-22Add py-botocore 1.13.44 (#14272)Adam J. Stewart1-2/+3
2019-12-22Add new release and remove extraneous patches (#14273)Piotr Luszczek1-2/+3
2019-12-22Set full xiar path when building cp2k with intel compiler (#14268)Glenn Johnson1-1/+2
2019-12-22Updating preferred OpenMPI version to 3.1.5 (#14266)justbennet1-2/+2
2019-12-22openPMD-api: 0.10.3 (#14265)Axel Huebl1-0/+1
Add latest release.
2019-12-21new module beagle (#14257)xfzhao2-0/+47
2019-12-21Bump py-slepc4py version, update url (#14264)Sajid Ali2-4/+12
* version bump modified: var/spack/repos/builtin/packages/py-slepc4py/package.py * slepc: update URL slepc4py: add 3.11.0 and update maintainers Co-authored-by: Satish Balay <balay@mcs.anl.gov>
2019-12-21use `sys.executable` instead of `python` in `_source_single_file` (#14252)Sajid Ali1-1/+1
2019-12-20build bazel with jdk (#14258)Sajid Ali1-1/+4
* build bazel with jdk * Flake 8 * Fix typo
2019-12-20microarchitectures: fix ppc flags for clang (#14196)Massimiliano Culpo2-18/+10
2019-12-20npm: add Python 3 support (#14255)Adam J. Stewart1-2/+33
2019-12-20Fetching from URLs falls back to mirrors if they exist (#13881)Massimiliano Culpo66-218/+321
Users can now list mirrors of the main url in packages. - [x] Instead of just a single `url` attribute, users can provide a list (`urls`) in the package, and these will be tried by in order by the fetch strategy. - [x] To handle one of the most common mirror cases, define a `GNUMirrorPackage` mixin to handle all the standard GNU mirrors. GNU packages can set `gnu_mirror_path` to define the path within a mirror, and the mixin handles setting up all the requisite GNU mirror URLs. - [x] update all GNU packages in `builtin` to use the `GNUMirrorPackage` mixin.
2019-12-20PGMATH library symbols patch (#14254)Nichols A. Romero2-0/+97
* Add symbols patch * Apply symbols patch to pgmath * Add github issue number for symbols patch. * Add naromero77 as a maintainer. * Patch only applied to March 2019 release and master.
2019-12-20Add py-pyinstrument package (#14241)Adam J. Stewart1-0/+21
2019-12-20node-js: add Python 3 support (#14242)Adam J. Stewart1-28/+52
* node-js: add Python 3 support * Update node-js, fix Python 3 support in v12
2019-12-20Latest versions of CMake need a hand for Python detection (#14247)Hadrien G1-0/+1
2019-12-20root: Record that old versions of ROOT don't support modern GCC (#14250)Hadrien G1-0/+3
* Record that old versions of ROOT don't support modern GCC * Well, actually I don't know about 6.07 * Fix typo and follow odd version recommendation from @chissg
2019-12-20QE Update Dec 2019 (#14238)Nichols A. Romero1-9/+23
* Add QE 6.5 * Support for serial HDF5 case with serial (no mpi) QE is now supported but requires a patch for 6.4.1 and 6.5. * Add naromero77 as a maintainer.
2019-12-20Do not apply the unuran patch to ROOT 6.06/xy, it is not compatible (#14245)Hadrien G1-1/+1
2019-12-20Add py-pyinstrument-cext package (#14240)Adam J. Stewart1-0/+18
2019-12-20Adding myself as a ROOT maintainer (#14251)Hadrien G1-1/+1
2019-12-20Added hwloc dependency to gromacs (#14244)Andras Wacha1-0/+7
2019-12-20R: added v3.6.2 (#14248)Patrick Schratz1-0/+1
2019-12-20ACTS: added v0.13.0 and v0.12.1 (#14246)Hadrien G1-0/+2
2019-12-19New package: cinema_lib (#14231)Ethan Stam1-0/+41
* Start cinema package * Remove boilerplate and add description * Formatting for pep8 * Correct milestone tag * 'master' instead of 'develop' Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Two variants, both with numpy and other small changes * When +image for scikit Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2019-12-19Add missing __init__.py under test, and correct bad file name from #13889. ↵Chris Green2-0/+4
(#14228)
2019-12-19root: Add chissg as maintainer. (#14235)Chris Green1-0/+2
2019-12-19Add py-scipy 1.4.1 (#14230)Adam J. Stewart1-2/+4
2019-12-19dmd: setup dependent build environment (#14198)Adam J. Stewart1-0/+3
* dmd: setup dependent build environment * Fix call signature of setup_dependent_build_environment
2019-12-19Remove python2-ism in root package (#14223)Hadrien G1-2/+1
2019-12-18package_sanity: add a test to enforce no nonexisting dependencies in builtinTodd Gamblin1-0/+16
We shouldn't allow packages to have missing dependencies in the mainline. - [x] Add a test to enforce this.
2019-12-18possible_dependencies() now reports missing dependenciesTodd Gamblin3-15/+69
- Add an optional argument so that `possible_dependencies()` will report missing dependencies. - Add a test to ensure it works. - Ignore missing dependencies in `possible_dependencies()` by default.
2019-12-18package: add spack.package.possible_dependencies methodTodd Gamblin2-4/+53
- this version allows getting possible dependencies of multiple packages or specs at once. - New method handles calling `PackageBase.possible_dependencies` multiple times and passing `visited` dict around.
2019-12-18performance: speed up `spack find` in environmentsTodd Gamblin1-7/+11
`Environment.added_specs()` has a loop around calls to `Package.installed()`, which can result in repeated DB queries. Optimize this with a read transaction in `Environment`.
2019-12-18performance: `spack spec` should use a read transacction with -ITodd Gamblin1-9/+26
`spack spec -I` queries the database for installation status and should use a read transaction around calls to `Spec.tree()`.
2019-12-18concretization: improve performance by avoiding database locksTodd Gamblin1-4/+6
Checks for deprecated specs were repeatedly taking out read locks on the database, which can be very slow. - [x] put a read transaction around the deprecation check
2019-12-18performance: memoize spack.architecture.get_platform()Todd Gamblin2-0/+8
`get_platform()` is pretty expensive and can be called many times in a spack invocation. - [x] memoize `get_platform()`