summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2017-04-23Use six.moves.input instead of raw_input (#3961)Adam J. Stewart2-3/+4
* Use six.moves.input instead of raw_input * Remove comment mentioning raw_input
2017-04-22External packages are now registered in the DB (#1167)Massimiliano Culpo15-57/+230
* treats correctly a change from `explicit=False` to `explicit=True` in an external package DB entry. * added unit tests * fixed issues raised by @tgamblin . In particular the PR is no more hash-changing for packages that are not external. * added a test to check correctness of a spec/yaml round-trip for things that involve an external * Don't find external module path at each step of concretization * it's not necessary.. The paths are retrieved at the end of concretizaion * Don't find replacements for external packages. * Test root of the DAG if external * No reason not to test if the root of the DAG is external when external packages are now first class citizens! * Create `external` property for Spec (for external_path and external_module) * Allow users to specify external package paths relative to spack * Canonicalize external package paths so that users may specify their locations relative to spack's directory. * Update tests to use new external_path and external properly. * skip license hooks on external
2017-04-22Find more versions from GitHub (#3952)Adam J. Stewart1-1/+1
2017-04-22add Intel mpi package and add MPI wrappers to Intel parallel studio (#3905)Denis Davydov1-0/+1
* intel-mpi: add new package * fix hashes * fix typo * flake8 * add install * blank line * error * add bin64 * fix MPI wrappers in intel-parallel-studio * add missing I_MPI_FC * use shorter hashes
2017-04-21PythonPackage builds flat installs instead of egg directories.Todd Gamblin1-1/+18
- Spack doesn't need eggs -- it manages its own directories - Simplify install layout and reduce sys.path searches by installing all packages flat (eggs are deprecated for wheels, and this is also what wheels do). - We now supply the --single-version-externally-managed argument to `setup.py install` for setuptools packages and setuptools. - modify packages to only use setuptools args if setuptools is an immediate dependency - Remove setuptools from packages that do not need it. - Some packages use setuptools *only* when certain args (likeb 'develop' or 'bdist') are supplied to setup.py, and they specifically do not use setuptools for installation. - Spack never calls setup.py this way, so just removing the setuptools dependency works for these packages.
2017-04-21fetch git submodules recursively (#3779)Christoph Junghans2-5/+16
* fetch git submodules recursively This is useful if the submodules have submodules themselves. On the other hand doing a recursive update doesn't hurt if there is only one level. * fetch submodules with depth=1 as well (fix #2190) * use git submodule with depth only for git>=1.8.4
2017-04-21spack uninstall no longer requires a known package. (#3915)Todd Gamblin6-82/+102
- Spack install would previously fail if it could not load a package for the thing being uninstalled. - This reworks uninstall to handle cases where the package is no longer known, e.g.: a) the package has been renamed or is no longer in Spack b) the repository the package came from is no longer registered in repos.yaml
2017-04-21Fix ABI detection issues with macOS gcc. (#3854)Todd Gamblin4-7/+23
- gcc on macOS says it's version 4.2.1, but it's really clang, and it's actually the *same* clang as the system clang. - It also doesn't respond with a full path when called with --print-file-name=libstdc++.dylib, which is expected from gcc in abi.py. Instead, it gives a relative path and _gcc_compiler_compare doesn't understand what to do with it. This results in errors like: ``` lib/spack/spack/abi.py, line 71, in _gcc_get_libstdcxx_version libpath = os.readlink(output.strip()) OSError: [Errno 2] No such file or directory: 'libstdc++.dylib' ``` - This commit does two things: 1. Ignore any gcc that's actually clang in abi.py. We can probably do better than this, but it's not clear there is a need to, since we should handle the compiler as clang, not gcc. 2. Don't auto-detect any "gcc" that is actually clang anymore. Ignore it and expect people to use clang (which is the default macOS compiler anyway). Users can still add fake gccs to their compilers.yaml if they want, but it's discouraged.
2017-04-21Fix checksumming in Python3; add more fetch tests (#3941)Todd Gamblin7-59/+151
* Checksum code wasn't opening binary files as binary. - Fixes Python 3 issue where files are opened as unicode text by default, and decoding fails for binary blobs. * Simplify fetch test parametrization. * - add tests for URL fetching and checksumming. - fix coverage on interface functions in FetchStrategy superclass - add some extra crypto tests.
2017-04-21Fix HPL build, convert to MakefilePackage (#3777)Adam J. Stewart1-0/+49
* Fix HPL build, convert to MakefilePackage * Flake8 fix * Fix: spec -> self.spec * Properly query for system libraries * Update Intel-MKL as well * Recurse in system libs, fix MKL path, fixes lapack_libs
2017-04-20Add Napoleon extension to support Google docstrings (#3920)Adam J. Stewart1-0/+1
2017-04-20Revert "Override partial installs by default" (#3918)Todd Gamblin4-203/+7
* Revert "Override partial installs by default (#3530)" This reverts commit a65c37f15dff4b4d60784fd4fcc55874ce9d6d11.
2017-04-20test/python_version.py: ported to pytest (#3438)Massimiliano Culpo1-79/+85
2017-04-19Override partial installs by default (#3530)scheibelp4-7/+203
* Package install remove prior unfinished installs Depending on how spack is terminated in the middle of building a package it may leave a partially installed package in the install prefix. Originally Spack treated the package as being installed if the prefix was present, in which case the user would have to manually remove the installation prefix before restarting an install. This commit adds a more thorough check to ensure that a package is actually installed. If the installation prefix is present but Spack determines that the install did not complete, it removes the installation prefix and starts a new install; if the user has enabled --keep-prefix, then Spack reverts to its old behavior. * Added test for partial install handling * Added test for restoring DB * Style fixes * Restoring 2.6 compatibility * Relocated repair logic to separate function * If --keep-prefix is set, package installs will continue an install from an existing prefix if one is present * check metadata consistency when continuing partial install * Added --force option to make spack reinstall a package (and all dependencies) from scratch * Updated bash completion; removed '-f' shorthand for '--force' for install command * dont use multiple write modes for completion file
2017-04-19Set proper deptypes for certain build systems (#3866)Adam J. Stewart3-4/+10
* Set proper deptypes for certain build systems * Add depends_on to namespace
2017-04-19Rework Spack's Mercurial support (#3834)Adam J. Stewart4-8/+59
* Add tests to mercurial package * Add support for --insecure with mercurial fetching * Install man pages and tab-completion scripts * Add tests and latest version for all deps * Flake8 fix * Use certifi module to find CA certificate * Flake8 fix * Unset PYTHONPATH when running hg * svn_fetch should use to svn-test, not hg-test * Drop Python 3 support in Mercurial Python 3 support is a work in progress and isn't currently recommended: https://www.mercurial-scm.org/wiki/SupportedPythonVersions * Test both secure and insecure hg fetching * Test both secure and insecure git and svn fetching
2017-04-15Fix flake8 changed files detection (#3855)Adam J. Stewart1-1/+1
2017-04-15Allow users to set parallel jobs in config.yaml (#3812)Adam J. Stewart5-6/+30
* Allow users to set parallel jobs in config.yaml * Undo change from endash to emdash * Remove parallel config, rename jobs to build_jobs
2017-04-14stage: remove a rogue period (#3846)Ben Boeckel1-1/+1
2017-04-14Add documentation on explicit param for do_install (#3823)Adam J. Stewart1-13/+15
2017-04-14Quick fix for #3153 (#3822)Mario Melara1-0/+3
2017-04-12Document commands that no longer work (#3781)Adam J. Stewart1-4/+37
2017-04-10doc: start with known issues section (#3722)Denis Davydov2-0/+38
2017-04-10Make it more clear that version ranges are inclusive (#3759)Adam J. Stewart1-43/+84
2017-04-07Abinit: a few modernizations for the package and its dependencies (#3699)Massimiliano Culpo1-5/+1
* libxc: added libs interface * hdf5: added libs interface, added conflicts * abinit: modernized package to use build interface * netcdf-fortran: added libs interface * abinit: added version 8.2.2
2017-04-06set_executable can set S_IX{GRP,OTH} (#3742)George Hartzell1-2/+8
`set_executable` now checks if a user/group.other had read permission on a file and if it does then it sets the corresponding executable bit. See #1483.
2017-04-06Correct uninstall -d to uninstall -R in tutorial (#3740)Max Katz1-2/+2
-d was changed to -R as of PR #1917
2017-04-05Update copyright on the docs. (#3718)Todd Gamblin1-2/+2
2017-04-05spack create MakefilePackage (#3710)Adam J. Stewart2-26/+38
* spack create MakefilePackage * Change default Perl template to match other build systems
2017-04-05Merge pull request #2789 from ↵scheibelp6-22/+112
paulhopkins/features/allow_package_installation_directory_to_be_configured Allow installation directory layout to be configured
2017-04-05Allow installation directory layout to be configured using either hashPaul Hopkins6-22/+112
length parameter or spec formatting.
2017-04-04Avoid null reference in arch concretization (#2596)scheibelp1-3/+18
Fixes #2587 The concretizer falls back on using the root architecture (followed by the default system architecture) to fill in unspecified arch properties for a spec. It failed to check cases where the root could be None.
2017-04-04Fixes #3675: Abinit: invalid spec on concretization #(3686)Massimiliano Culpo1-1/+1
2017-04-04Update getting_started.rst (#3685)Justin Cook1-1/+1
Fixed typo referring to .bashrc file
2017-04-03Overhaul Spack's URL parsing (#2972)Adam J. Stewart14-565/+1166
* Remove fake URLs from Spack * Ignore long lines for URLs that start with ftp: * Preliminary changes to version regexes * New redesign of version regexes * Allow letters in version-only * Fix detection of versions that end in Final * Rearrange a few regexes and add examples * Add tests for common download repositories * Add test cases for common tarball naming schemes * Finalize version regexes * spack url test -> spack url summary * Clean up comments * Rearrange suffix checks * Use query strings for name detection * Remove no longer necessary url_for_version functions * Strip off extraneous information after package name * Add one more test * Dot in square brackets does not need to be escaped * Move renaming outside of parse_name_offset * Fix versions for a couple more packages * Fix flake8 and doc tests * Correctly parse Python, Lua, and Bio++ package names * Use effective URLs for mfem * Add checksummed version to mitos * Remove url_for_version from STAR-CCM+ package * Revert changes to version numbers with underscores and dashes * Fix name detection for tbb * Correctly parse Ruby gems * Reverted mfem back to shortened URLs. * Updated instructions for better security * Remove preferred=True from newest version * Add tests for new `spack url list` flags * Add tests for strip_name_suffixes * Add unit tests for version separators * Fix bugs related to parseable name but in parseable version * Remove dead code, update docstring * Ignore 'binary' at end of version string * Remove platform from version * Flip libedit version numbers * Re-support weird NCO alpha/beta versions * Rebase and remove one new fake URL * Add / to beginning of regex to avoid picking up similarly named packages * Ignore weird tar versions * Fix bug in url parse --spider when no versions found * Less strict version matching for spack versions * Don't rename Python packages * Be a little more selective, version must begin with a digit * Re-add fake URLs * Fix up several other packages * Ignore more file endings * Add parsing support for Miniconda * Update tab completion * XFAILS are now PASSES for 2 web tests
2017-04-02new directive: conflicts() (#3125)Massimiliano Culpo4-1/+99
* Add conflicts(<spec>) directive * openblas: added conflicts for intel@16 refs #3119 * added brief docs and unit tests
2017-04-01Add epigraph in docs.Todd Gamblin1-1/+5
2017-04-01Add better tests for web.py; fix some bugs found with spidering.Todd Gamblin8-35/+283
- _spider in web.py was actually failing to spider deeper than a certain point. - Fixed multiprocessing pools to not use daemons and to allow recursive spawning. - Added detailed tests for spidering and for finding archive versions. - left some xfail URL finding exercises for the reader. - Fix noqa annotations for some @when decorators
2017-04-01Fix Python 3 support in spack versionsAdam J. Stewart2-4/+5
- Add missing import, fixes spack versions in Python 2 - Fix spack versions in Python 3
2017-04-01fix annoying 'fatal: Not a git repository' error message (#3657)Kenneth Hoste2-4/+8
fix annoying 'fatal: Not a git repository' error message produced by 'spack list' when Spack is not run from a Git repository (#3657)
2017-03-31fix flake8 ignore syntax (use E instead of ignore) (#3651)Gregory Lee1-1/+1
2017-03-31test/package_sanity.py: ported to pytest (#3474)Massimiliano Culpo1-34/+33
2017-03-31test/versions.py: ported to pytest (#3635)Massimiliano Culpo1-403/+443
2017-03-31Spack works with Python 3Todd Gamblin2-2/+2
- Update version guard in spack script to allow python 3 - Update min required version in the docs
2017-03-31Clean up tests and add Python3 to Travis.Todd Gamblin2-89/+155
- Clean up spec_syntax tests: don't dependend on DB order. - spec_syntax hash parsing tests were strongly dependent on the order the DB was traversed. - Tests now specifically grab the specs they want from the mock DB. - Tests are more readable as a result. - Add Python3 versions to Travis tests.
2017-03-31Fix concretization bugs with virtuals and deptypes.Todd Gamblin2-25/+39
1. Fix #2807: Can't depend on virtual and non-virtual package - This is tested by test_my_dep_depends_on_provider_of_my_virtual_dep in the concretize.py test. - This was actually working in the test suite, but it depended on the order the dependencies were resolved in. Resolving non-virtual then virtual worked, but virtual, then non-virtual did not. - Problem was that an unnecessary copy was made of a spec that already had some dependencies set up, and the copy lost half of some of the dependency relationships. This caused the "can'd depend on X twice error". - Fix by eliminating unnecessary copy and ensuring that dep parameter of _merge_dependency is always safe to own -- i.e. it's a defensive copy from somewhere else. 2. Fix bug and simplify concretization of deptypes. - deptypes weren't being accumulated; they were being set on each DependencySpec. This could cause concretization to get into an infinite loop. - Fixed by accumulating deptypes in DependencySpec.update_deptypes() - Also simplified deptype normalization logic: deptypes are now merged in constrain() like everything else -- there is no need to merge them specially or to look at dpeendents in _merge_dependency(). - Add some docstrings to deptype tests.
2017-03-31Use key sorting instead of cmp()Todd Gamblin19-396/+318
- Get rid of pkgsort() usage for preferred variants. - Concretization is now entirely based on key-based sorting. - Remove PreferredPackages class and various spec cmp() methods. - Replace with PackagePrefs class that implements a key function for sorting according to packages.yaml. - Clear package pref caches on config test. - Explicit compare methods instead of total_ordering in Version. - Our total_ordering backport wasn't making Python 3 happy for some reason. - Python 3's functools.total_ordering and spelling the operators out fixes the problem. - Fix unicode issues with spec hashes, json, & YAML - Try to use str everywhere and avoid unicode objects in python 2.
2017-03-31Resolve Python2/Python3 unicode issues by using str()Todd Gamblin7-13/+15
- Remove ascii encoding assumption from spack_yaml - proc.communicate() returns bytes; convert to str before adding. - Fix various byte string/unicode issues for Python 2/3 support - Need to decode subprocess output as utf-8 in from_sourcing_files. - Fix comments in strify()
2017-03-31Convert Python 2 idioms to Python 2/3-compatible ones.Todd Gamblin73-321/+392
- convert print, StringIO, except as, octals, izip - convert print statement to print function - convert StringIO to six.StringIO - remove usage of csv reader in Spec, in favor of simple regex - csv reader only does byte strings - convert 0755 octal literals to 0o755 - convert `except Foo, e` to `except Foo as e` - fix a few places `str` is used. - may need to switch everything to str later. - convert iteritems usages to use six.iteritems - fix urllib and HTMLParser - port metaclasses to use six.with_metaclass - More octal literal conversions for Python 2/3 - Fix a new octal literal. - Convert `basestring` to `six.string_types` - Convert xrange -> range - Fix various issues with encoding, iteritems, and Python3 semantics. - Convert contextlib.nested to explicitly nexted context managers. - Convert use of filter() to list comprehensions. - Replace reduce() with list comprehensions. - Clean up composite: replace inspect.ismethod() with callable() - Python 3 doesn't have "method" objects; inspect.ismethod returns False. - Need to use callable in Composite to make it work. - Update colify to use future division. - Fix zip() usages that need to be lists. - Python3: Use line-buffered logging instead of unbuffered. - Python3 raises an error with unbuffered I/O - See https://bugs.python.org/issue17404
2017-03-31Update externals to work with Python 3Todd Gamblin42-131/+7062
- Update YAML version to support Python 3 - Python 3 support for ordereddict backport - Exclude Python3 YAML from version tests. - Vendor six into Spack. - Make Python version-check tests work with Python 3 - Add ability to add version check exceptions with '# nopyqver' line comments.