summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2019-12-31add new package : opentsdb (#14317)darmac1-0/+33
* add new package : opentsdb * remove redundant url_for_version()
2019-12-31New package - zziplib (#14318)Glenn Johnson2-0/+743
This PR adds the zziplib package.
2019-12-31new package: py-asciitree (#14327)Sinan1-0/+17
* new package: py-asciitree * fix typo
2019-12-31Add OpenFOAM 1912 (#14322)Mark Olesen1-0/+1
2019-12-30copyright: update copyright dates for 2020 (#14328)Todd Gamblin4346-4351/+4351
2019-12-30bugfix: add required fixture for CDash authentication test (#14325)Todd Gamblin1-1/+1
2019-12-30Add support for authenticated CDash uploads (#14200)Zack Galbreath4-3/+35
2019-12-30refactor: rename mock_config fixture to mock_low_high_configTodd Gamblin4-31/+32
This avoids confusion with mock_configuration.
2019-12-30argparse: lazily construct common argumentsTodd Gamblin3-72/+120
Continuing to shave small bits of time off startup -- `spack.cmd.common.arguments` constructs many `Args` objects at module scope, which has to be done for all commands that import it. Instead of doing this at load time, do it lazily. - [x] construct Args objects lazily - [x] remove the module-scoped argparse fixture - [x] make the mock config scope set dirty to False by default (like the regular scope) This *seems* to reduce load time slightly
2019-12-30tests: refactor tests to avoid persistent global stateTodd Gamblin16-90/+141
Previously, fixtures like `config`, `database`, and `store` were module-scoped, but frequently used as test function arguments. These fixtures swap out global on setup and restore them on teardown. As function arguments, they would do the right set-up, but they'd leave the global changes in place for the whole module the function lived in. This meant that if you use `config` once, other functions in the same module would inadvertently inherit the mock Spack configuration, as it would only be torn down once all tests in the module were complete. In general, we should module- or session-scope the *STATE* required for these global objects (as it's expensive to create0, but we shouldn't module-or session scope the activation/use of them, or things can get really confusing. - [x] Make generic context managers for global-modifying fixtures. - [x] Make session- and module-scoped fixtures that ONLY build filesystem state and create objects, but do not swap out any variables. - [x] Make seeparate function-scoped fixtures that *use* the session scoped fixtures and actually swap out (and back in) the global variables like `config`, `database`, and `store`. These changes make it so that global changes are *only* ever alive for a singlee test function, and we don't get weird dependencies because a global fixture hasn't been destroyed.
2019-12-30tests: make env tests that use configs non-destructiveTodd Gamblin1-10/+10
Environment tests pushed config scopes but didn't properly remove them. - [x] use `with env:` context manager instead of `env.prepare_config_scopes()`
2019-12-30package_prefs: move class-level cache to PackagePref instanceTodd Gamblin5-57/+11
`PackagePrefs` has had a class-level cache of data from `packages.yaml` for a long time, but it complicates testing and leads to subtle errors, especially now that we frequently manipulate custom config scopes and environments. Moving the cache to instance-level doesn't slow down concretization or the test suite, and it just caches for the life of a `PackagePrefs` instance (i.e., for a single cocncretization) so we don't need to worry about global state anymore. - [x] Remove class-level caches from `PackagePrefs` - [x] Add a cached _spec_order object on each `PackagePrefs` instance - [x] Remove all calls to `PackagePrefs.clear_caches()`
2019-12-29Add py-numpy 1.16.6 (#14313)Adam J. Stewart1-0/+1
2019-12-29Add support for graphite2 support to harfbuzz (#14300)Glenn Johnson1-0/+5
This PR adds graphite2 support to harfbuzz. This PR depends on PR #14299, which added the graphite2 package to spack.
2019-12-28externals: avoid importing jinja2 on startup (#14308)Todd Gamblin2-5/+6
Jinja2 costs a tenth to a few tenths of a second to import, so we should avoid importing it on startup. - [x] only import jinja2 within functions
2019-12-28bugfix: colify_table should not revert to 1 column for non-tty (#14307)Todd Gamblin1-4/+13
Commands like `spack blame` were printig poorly when redirected to files, as colify reverts to a single column when redirected. This works for list data but not tables. - [x] Force a table by always passing `tty=True` from `colify_table()`
2019-12-28add new package : jafka (#14304)darmac1-0/+29
2019-12-28New package - TECkit (#14301)Glenn Johnson1-0/+29
This PR adds the teckit package to spack.
2019-12-28New package - graphite2 (#14299)Glenn Johnson1-0/+18
* New package - graphite2 This PR adds the Graphite2 package to Spack. * Replace smart quotes with standard quotes
2019-12-27setup-env-test: fix pipe redirect (#14306)Johannes Blaschke1-1/+1
2019-12-27Improve info variant header (#14275)Dr. Christian Tacke1-1/+3
In "spack info" the Variants header currently has two blank lines under it. That's too much. It looks like the actual content belongs to something else. Instead underline the headers to make things more obvious.
2019-12-27New package - libpaper (#14302)Glenn Johnson1-0/+22
This PR adds the paper library to spack.
2019-12-27added build dependences required after cairo package change (#14297)Gregory Lee1-1/+1
* added build dependences required after cairo package change * fixed pango build dependencies
2019-12-26patch cairo build files for build without fontconfig fixes #14112 (#14296)Gregory Lee2-0/+49
* patch cairo build files for build without fontconfig * added build dependencies for cairo
2019-12-26Add Avizo Recipe (#12792)asmaahassan901-0/+52
* Add Avizo Recipe * make changes as per review * fix home url and linting * Fix url * fix identation * change checksum to sha256 instead of md5 * fix installation * fix lint * fix identation * make it compatible with python 2.6 * enhancing recipe and fixing avizo licensing changes as per suggestions from reviewer; fix licensing * fix identation * use new setup_run_environment function
2019-12-26aspa: Fix common errors. (#14295)t-karatsu2-0/+61
* aspa: Fix common errors. * defined new member 'Vec_value_type' in 'self' namespace. * fix narrowing error. * fix patch about typecast.
2019-12-25Migrate build tests from Travis to Github Actions (#13967)Massimiliano Culpo4-50/+74
This PR moves build smoke tests from TravisCI and migrates them to Github Actions. The result is that build tests are performed in parallel with unit tests and they don't hog additional resources on Travis. The workflow will not run if a PR only changes packages in the built-in repository, but will always run on pushes to develop or master. * Removed build tests from Travis and passed them to Github Actions * Store ~/.ccache in Github Actions cache * Add filters on paths and make sure this workflow don't run * Use paths-ignore and exclude only files in the built-in repo * Added a badge to README.md
2019-12-24tests: finish removing pyqver from the repository (#14294)Todd Gamblin3-6/+2
Remove a few remaining mentions of the pyqver package, which was removed in #14289.
2019-12-24tests: check min required python version with vermin (#14289)Massimiliano Culpo13-769/+42
This commit removes the `python_version.py` unit test module and the vendored dependencies `pyqver2.py` and `pyqver3.py`. It substitutes them with an equivalent check done using `vermin` that is run as a separate workflow via Github Actions. This allows us to delete 2 vendored dependencies that are unmaintained and substitutes them with a maintained tool. Also, updates the list of vendored dependencies.
2019-12-24a64fx: fix typo in GCC flags (#14286)t-karatsu1-2/+2
2019-12-24intel-pin: add version 3.11 (#14292)Matthias Diener1-0/+1
2019-12-24cmake: add version 3.16.2 (#14291)Matthias Diener1-0/+1
2019-12-24scons: add version 3.1.2 (#14290)Matthias Diener1-0/+1
2019-12-24Bump nektar to 5.0.0. (#14288)健美猞猁1-0/+1
2019-12-24Merge branch 'releases/v0.13' into developTodd Gamblin2-1/+17
2019-12-23update CHANGELOG.md for 0.13.3v0.13.3Todd Gamblin1-0/+16
2019-12-23version bump: 0.13.3Todd Gamblin1-1/+1
2019-12-23Travis exits at the first failing test, pin codecov at v4.5.4 (#14179)Massimiliano Culpo2-2/+2
Before this commit we used to run the entire unit test suite in the presence of a failure. Since we currently rely a lot on the state of the filesystem etc. the end report was most of the time showing spurious failures that were a consequence of the first failing test. This PR makes unit tests exit at the first failing test Also, pin codecov at v4.5.4 (last one supporting Python 2.6)
2019-12-23performance: dont' read `spec.yaml` files twice in view regenerationTodd Gamblin2-4/+8
`ViewDescriptor.regenerate()` calls `get_all_specs()`, which reads `spec.yaml` files, which is slow. It's fine to do this once, but `view.remove_specs()` *also* calls it immediately afterwards. - [x] Pass the result of `get_all_specs()` as an optional parameter to `view.remove_specs()` to avoid reading `spec.yaml` files twice.
2019-12-23performance: don't recompute hashes when regenerating environmentsTodd Gamblin2-4/+9
`ViewDescriptor.regenerate()` was copying specs and stripping build dependencies, which clears `_hash` and other cached fields on concrete specs, which causes a bunch of YAML hashes to be recomputed. - [x] Preserve the `_hash` and `_normal` fields on stripped specs, as these will be unchanged.
2019-12-23performance: reduce system calls required for remove_dead_linksTodd Gamblin1-4/+2
`os.path.exists()` will report False if the target of a symlink doesn't exist, so we can avoid a costly call to realpath here.
2019-12-23performance: only regenerate env views once in `spack install`Todd Gamblin2-9/+29
`spack install` previously concretized, writes the entire environment out, regenerated views, then wrote and regenerated views again. Regenerating views is slow, so ensure that we only do that once. - [x] add an option to env.write() to skip view regeneration - [x] add a note on whether regenerate_views() shouldn't just be a separate operation -- not clear if we want to keep it as part of write to ensure consistency, or take it out to avoid performance issues.
2019-12-23performance: add read transactions for `install_all()` and `install()`Todd Gamblin1-31/+35
Environments need to read the DB a lot when installing all specs. - [x] Put a read transaction around `install_all()` and `install()` to avoid repeated locking
2019-12-23lock transactions: avoid redundant reading in write transactionsTodd Gamblin2-1/+62
Our `LockTransaction` class was reading overly aggressively. In cases like this: ``` 1 with spack.store.db.read_transaction(): 2 with spack.store.db.write_transaction(): 3 ... ``` The `ReadTransaction` on line 1 would read in the DB, but the WriteTransaction on line 2 would read in the DB *again*, even though we had a read lock the whole time. `WriteTransaction`s were only considering nested writes to decide when to read, but they didn't know when we already had a read lock. - [x] `Lock.acquire_write()` return `False` in cases where we already had a read lock.
2019-12-23lock transactions: ensure that nested write transactions writeTodd Gamblin2-1/+64
If a write transaction was nested inside a read transaction, it would not write properly on release, e.g., in a sequence like this, inside our `LockTransaction` class: ``` 1 with spack.store.db.read_transaction(): 2 with spack.store.db.write_transaction(): 3 ... 4 with spack.store.db.read_transaction(): ... ``` The WriteTransaction on line 2 had no way of knowing that its `__exit__()` call was the last *write* in the nesting, and it would skip calling its write function. The `__exit__()` call of the `ReadTransaction` on line 1 wouldn't know how to write, and the file would never be written. The DB would be correct in memory, but the `ReadTransaction` on line 4 would re-read the whole DB assuming that other processes may have modified it. Since the DB was never written, we got stale data. - [x] Make `Lock.release_write()` return `True` whenever we release the *last write* in a nest.
2019-12-23lock transactions: fix non-transactional writesTodd Gamblin4-172/+285
Lock transactions were actually writing *after* the lock was released. The code was looking at the result of `release_write()` before writing, then writing based on whether the lock was released. This is pretty obviously wrong. - [x] Refactor `Lock` so that a release function can be passed to the `Lock` and called *only* when a lock is really released. - [x] Refactor `LockTransaction` classes to use the release function instead of checking the return value of `release_read()` / `release_write()`
2019-12-23performance: avoid repeated DB locking on view generationTodd Gamblin1-10/+13
`ViewDescriptor.regenerate()` checks repeatedly whether packages are installed and also does a lot of DB queries. Put a read transaction around the whole thing to avoid repeatedly locking and unlocking the DB.
2019-12-23performance: speed up `spack find` in environmentsTodd Gamblin1-7/+11
`Environment.added_specs()` has a loop around calls to `Package.installed()`, which can result in repeated DB queries. Optimize this with a read transaction in `Environment`.
2019-12-23performance: `spack spec` should use a read transacction with -ITodd Gamblin1-9/+26
`spack spec -I` queries the database for installation status and should use a read transaction around calls to `Spec.tree()`.
2019-12-23concretization: improve performance by avoiding database locksTodd Gamblin1-4/+6
Checks for deprecated specs were repeatedly taking out read locks on the database, which can be very slow. - [x] put a read transaction around the deprecation check