summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2015-12-16Merge pull request #239 from LLNL/bugfix/238-db-specs-not-concreteGregory Lee3-4/+17
Fix #238: Database specs need to be concrete when read in as well.
2015-12-16minor comment tweak in modules.pyMike Nolta1-2/+2
2015-12-15Fix #238: Database specs need to be concrete when read in as well.Todd Gamblin3-4/+17
2015-12-14Fix #206: need to make deps AND root concrete when read in.Todd Gamblin1-2/+4
2015-12-11Make internal hash dep sort order match external one.Todd Gamblin1-1/+2
2015-12-11Fix #217: update spec_dag test for new `_cmp_key`.Todd Gamblin1-3/+5
2015-12-11Fix #217: Use MUCH faster hashing, reduce number of DAG copies.Todd Gamblin2-12/+18
This changes the hash algorithm so that it does much less object allocation and copying, and so that it is correct. The old version of `_cmp_key()` would call `sorted_deps`, which would call `flat_dependencies` to get a list of dependencies so that it could sort them in alphabetical order. This isn't necessary in the `_cmp_key()`, and in fact we want more DAG structure than that to be included in the `_cmp_key()`. The new version constructs a tuple without copying the Spec DAG, and the tuple contains hashes of sub-DAGs that are computed recursively in-place. This is way faster than the previous algorithm and reduces the numebr of copies significantly. It is also a correct DAG hash. Example timing and copy counts for the different hashing algorithms we've tried: Original (wrong) Spec hash: ``` 106,170 copies real 0m5.024s user 0m4.949s sys 0m0.104s ``` Spec hash using YAML `dag_hash()`: ``` 3,794 copies real 0m5.024s user 0m4.949s sys 0m0.104s New no-copy, no-YAML hash: ``` 3,594 copies real 0m2.543s user 0m2.435s sys 0m0.104s ``` So now we have a hash that is correct AND faster. The remaining ~3k copies happen mostly during concretization, and as all packages are initially loaded. I believe this is because Spack currently has to load all packages to figure out virtual dependency information; it could also be becasue there ar a lot of lookups of partial specs in concretize. I can investigate this further.
2015-12-11Fix #217: Make package cache use DAG hash instead of sorted deps.Todd Gamblin1-7/+8
- Gets rid of last vestige of old-style specs. - Uses new hashing for lookup
2015-12-09bugfix: doc build needed additional sys.path due to externals change.Todd Gamblin1-0/+1
2015-12-09Change github.com/scalability-llnl to github.com/llnl everywhere.Todd Gamblin139-144/+144
2015-11-27Fixed bug #42: problem with satisfies() for virtual dependencies.Todd Gamblin3-5/+24
- _cross_provider_maps() had suffered some bit rot (map returned was ill-formed but still worked for cases with one vdep) - ProviderIndex.satisfies() was only checking whether the result map was non-empty. It should check whether all common vdeps are *in* the result map, as that indicates there is *some* way to satisfy *all* of them. We were checking whether there was some way to satisfy *any one* of them, which is wrong. - Above would cause a problem when there is more than one vdep provider. - Added test that covers this case. - Added `constrained()` method to Spec. Analogous to `normalized()`: `constrain():constrained() :: normalize():normalized()`
2015-11-23Merge branch 'scheibelp-features/externaldeps' into developTodd Gamblin42-44/+44
2015-11-23Merge pull request #160 from SteVwonder/stevwonder/developTodd Gamblin1-2/+2
Fixed -l flag for 'spack extensions'
2015-11-20modified warning to reference spack cleanTom Scogland1-1/+1
clean --dist has been removed, command fixed to something that works.
2015-11-12Fixed -l flag for 'spack extensions'Stephen Herbein1-2/+2
2015-11-11Insert lib/spack/external into sys.path. This avoids cases where the systemPeter Scheibel42-44/+44
python install and lib/spack/external have the same library installed. This requires modifying the names of some modules in lib/spack/external in cases where both the system python and backported features of future python versions (i.e. after 2.6) are used (previously distinguished by "from external import X" and "import X").
2015-11-04Fix SPACK-93, SPACK-94, GitHub #150Todd Gamblin4-21/+18
- `remove_prefix` was modified to remove from the DB, but the package may not have been added to the DB yet when `remove_prefix` is called from `cleanup`. - Made `remove_prefix` a pure utility function (it just removes the prefix) - Added `installed_db.remove()` call only after the `remove_prefix` in `uninstall`.
2015-11-04Make architecture reflect OS *and* machine. Use Python's platform module.Todd Gamblin1-11/+8
2015-10-30Fixed typo : write_lock() -> write_transaction()Massimiliano Culpo3-3/+3
2015-10-29Merge Greg Becker's build database support into develop.Todd Gamblin22-139/+1641
2015-10-29Merge pull request #124 from scheibelp/features/testinstall-cmdTodd Gamblin6-3/+350
Features/testinstall cmd
2015-10-29Merge pull request #145 from markcmiller86/v-char-bugfixTodd Gamblin1-2/+2
generalize one of the cases to handle many more kinds of formatting o…
2015-10-29Merge pull request #139 from mathstuf/use-uname-for-archTodd Gamblin1-10/+10
architecture: use uname if available
2015-10-27generalize one of the cases to handle many more kinds of formatting optionsmiller861-2/+2
2015-10-27Build database working with simple transaction support; all tests passing.Todd Gamblin12-230/+645
2015-10-27Fix stupid lock bug.Todd Gamblin3-41/+53
- Code simplification ignored case where exception was raised. - If LockError was raised, read and write counts were incremented erroneously. - updated lock test.
2015-10-27Not all package stages have an archive file (e.g. source code repos) but all ofPeter Scheibel1-1/+1
them do have a source_path: use this instead to check whether the package resources were successfully retrieved.
2015-10-27Lock acquires return True/False depending on whether they got POSIX lock.Todd Gamblin1-4/+15
2015-10-26architecture: remove custom mac_type methodBen Boeckel1-13/+0
2015-10-26architecture: use uname if availableBen Boeckel1-1/+14
2015-10-26Fetch errors were also terminating runs of test-install with system exit, soPeter Scheibel3-1/+10
stage.fetch() was updated to raise a FetchError instead of calling tty.die(). Output is the same for spack install in case of a fetch error.
2015-10-26python: use the setdefault method on dictBen Boeckel3-16/+7
It allows more concise code and skips some key lookups.
2015-10-26Making SpackError reference consistent.Peter Scheibel1-2/+1
2015-10-26Automatically create a 'test-output' directory in the current directory if noPeter Scheibel1-1/+4
output path is specified. Test output files are placed in this directory. Furthermore the filenames now have the prefix "test" (but otherwise are the string representation of the spec ID as before).
2015-10-24Working Lock class, now uses POSIX fcntl locks, extensive unit test.Todd Gamblin3-84/+350
- llnl.util.lock now uses fcntl.lockf instead of flock - purported to have more NFS compatibility. - Added an extensive test case for locks. - tests acquiring, releasing, upgrading, timeouts, shared, & exclusive cases.
2015-10-24Add a multiprocess Barrier class to use for testing parallel code.Todd Gamblin1-1/+49
2015-10-23The HTML number conversion regex operating against a byte string will onlyPeter Scheibel1-1/+2
convert individual bytes, so therefore incorrectly converts utf-8 encoded characters. Decoding byte strings to unicode objects results in correct HTML number encodings.
2015-10-231. Rename CommandError -> InstallErrorPeter Scheibel2-5/+7
2. InstallError now subclasses SpackError vs. StandardError (so it is now handled by the spack shell script)
2015-10-221. Added CommandError exception to build_environmentPeter Scheibel2-15/+28
2. The parent of a failed child process in build_environment.fork no longer calls sys.exit - instead it raises a CommandError (from [1]) 3. test-install command now attempts to install all packages even if one fails
2015-10-22Generate test results (designated as skipped) for parents of failed dependenciesPeter Scheibel2-24/+45
2015-10-21Merge pull request #129 from mathstuf/separate-prefixTodd Gamblin1-6/+8
spack: split spack_root from prefix
2015-10-18Add Spack logo.Todd Gamblin1-3/+3
2015-10-17Better info variant output.Todd Gamblin1-3/+13
2015-10-15Created unit test for core logic in test-install command.Peter Scheibel3-12/+145
2015-10-15Better description for test-install commandPeter Scheibel1-1/+1
2015-10-15Minor edit for clarity (generate output for single top level spec vs. iteratingPeter Scheibel1-2/+1
through collection of size 1)
2015-10-15Originally I enforced specifying 1 top-level package with the test-installPeter Scheibel1-5/+4
command by having it consume exactly 1 positional argument (i.e. by removing "nargs=argparse.REMAINDER") but this does not work when configuring dependencies of a top-level package (which show up as additional positional args). Instead now there is an explicit check to ensure there is only 1 top-level package.
2015-10-151. Specifying the output file path for test-install is now an option (vs. anPeter Scheibel2-12/+22
argument). The default path is [package id].xml in the CWD where test-install is called from. 2. Fixed a bug with package.build_log_path (which was added in this branch). 3. keep_stage for package.do_install is now set. This allows uninstalling and reinstalling packages without (re) downloading them.
2015-10-15Move logic for tracking the build log into package.py (since that is what isPeter Scheibel2-9/+10
managing the build log) and expose as package.build_log_path.
2015-10-15Update test failure output: don't include the entire build log, just lines whichPeter Scheibel1-5/+10
mention errors (or if no such lines can be found, output the last 10 lines from the log).