summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2017-03-31Fix concretization bugs with virtuals and deptypes.Todd Gamblin2-25/+39
1. Fix #2807: Can't depend on virtual and non-virtual package - This is tested by test_my_dep_depends_on_provider_of_my_virtual_dep in the concretize.py test. - This was actually working in the test suite, but it depended on the order the dependencies were resolved in. Resolving non-virtual then virtual worked, but virtual, then non-virtual did not. - Problem was that an unnecessary copy was made of a spec that already had some dependencies set up, and the copy lost half of some of the dependency relationships. This caused the "can'd depend on X twice error". - Fix by eliminating unnecessary copy and ensuring that dep parameter of _merge_dependency is always safe to own -- i.e. it's a defensive copy from somewhere else. 2. Fix bug and simplify concretization of deptypes. - deptypes weren't being accumulated; they were being set on each DependencySpec. This could cause concretization to get into an infinite loop. - Fixed by accumulating deptypes in DependencySpec.update_deptypes() - Also simplified deptype normalization logic: deptypes are now merged in constrain() like everything else -- there is no need to merge them specially or to look at dpeendents in _merge_dependency(). - Add some docstrings to deptype tests.
2017-03-31Use key sorting instead of cmp()Todd Gamblin19-396/+318
- Get rid of pkgsort() usage for preferred variants. - Concretization is now entirely based on key-based sorting. - Remove PreferredPackages class and various spec cmp() methods. - Replace with PackagePrefs class that implements a key function for sorting according to packages.yaml. - Clear package pref caches on config test. - Explicit compare methods instead of total_ordering in Version. - Our total_ordering backport wasn't making Python 3 happy for some reason. - Python 3's functools.total_ordering and spelling the operators out fixes the problem. - Fix unicode issues with spec hashes, json, & YAML - Try to use str everywhere and avoid unicode objects in python 2.
2017-03-31Resolve Python2/Python3 unicode issues by using str()Todd Gamblin7-13/+15
- Remove ascii encoding assumption from spack_yaml - proc.communicate() returns bytes; convert to str before adding. - Fix various byte string/unicode issues for Python 2/3 support - Need to decode subprocess output as utf-8 in from_sourcing_files. - Fix comments in strify()
2017-03-31Convert Python 2 idioms to Python 2/3-compatible ones.Todd Gamblin73-321/+392
- convert print, StringIO, except as, octals, izip - convert print statement to print function - convert StringIO to six.StringIO - remove usage of csv reader in Spec, in favor of simple regex - csv reader only does byte strings - convert 0755 octal literals to 0o755 - convert `except Foo, e` to `except Foo as e` - fix a few places `str` is used. - may need to switch everything to str later. - convert iteritems usages to use six.iteritems - fix urllib and HTMLParser - port metaclasses to use six.with_metaclass - More octal literal conversions for Python 2/3 - Fix a new octal literal. - Convert `basestring` to `six.string_types` - Convert xrange -> range - Fix various issues with encoding, iteritems, and Python3 semantics. - Convert contextlib.nested to explicitly nexted context managers. - Convert use of filter() to list comprehensions. - Replace reduce() with list comprehensions. - Clean up composite: replace inspect.ismethod() with callable() - Python 3 doesn't have "method" objects; inspect.ismethod returns False. - Need to use callable in Composite to make it work. - Update colify to use future division. - Fix zip() usages that need to be lists. - Python3: Use line-buffered logging instead of unbuffered. - Python3 raises an error with unbuffered I/O - See https://bugs.python.org/issue17404
2017-03-31Update externals to work with Python 3Todd Gamblin42-131/+7062
- Update YAML version to support Python 3 - Python 3 support for ordereddict backport - Exclude Python3 YAML from version tests. - Vendor six into Spack. - Make Python version-check tests work with Python 3 - Add ability to add version check exceptions with '# nopyqver' line comments.
2017-03-31PythonPackage: Let There Be Tests! (#2869)Adam J. Stewart2-7/+75
* Run python setup.py test if --run-tests * Attempt to import the Python module after installation * Add testing support to numpy and scipy * Remove duplicated comments * Update to new run-tests callback methodology * Remove unrelated changes for another PR
2017-03-31Fix mxml (#3639)Michael Kuhn1-0/+1
mxml is now hosted on GitHub.
2017-03-30Extendable Perl (#3614)Milton Woods8-7/+179
* perl: make extendable and add Module::Build package * perl: allow 'spack create' to identify perl packages from their contents * perl-module-build: fix indenting of package docstring * perl: split install() method for extensions into phases * perl: auto-detect build method (Makefile.PL vs Build.PL) and define a 'check' method * PerlPackage: use import statements similar to those in AutotoolsPackage * PerlModule: fix detection of Build.PL * PerlPackageTemplate: remove extraneous lines to avoid flake8 warnings * PerlPackageTemplate: split into separate templates for Makefile.PL and Build.PL * PerlPackage: add cross-references to docstrings * AutotoolsPackage: fix ambiguous cross-references to avoid errors in doc tests * PerlbuildPackageTemplate: depend on perl-module-build if Build.PL exists
2017-03-28Fix bug in `spack find` for installations with unknown namespaces. (#3573)Todd Gamblin1-1/+7
- Spack find would fail with "unknown namespace" for some queries when a package from an unknown namespace was installed. - Solve by being conservative: assume unknown packages are NOT providers of virtual dependencies.
2017-03-28Bugfix: allow deactivating installs that don't have packages anymore. (#3572)Todd Gamblin1-5/+4
- deactivate -a wouldn't work if the installation's package was no longer available. - Fix installed_extensions_for so that it doesn't need to look at the package.py file.
2017-03-24make git fetching quite (#3180)Denis Davydov1-12/+31
also output repo and branch/commit/tag in one line
2017-03-23Add Expect package (#3517)Adam J. Stewart1-1/+2
* Add Expect package * Ignore patches during flake8 tests for package.py files * Remove controversial changes
2017-03-21Fix for `find --explicit` #3374 (#3492)George Hartzell1-1/+1
This fixes the problem described in #3374, which describes `spack find` ignore explicit/implicit. I believe that this was broken in #2626. This restores the behavior of implicit/explicit for me. I believe that it does not screw anything else up, but ....
2017-03-20Improve output for compiler commands (#3480)scheibelp2-1/+14
* Order listed compiler sections "spack compiler list" output compiler sections in an arbitrary order. With this commit compiler sections are ordered primarily by compiler name and then by operating system and target. * Compiler search lists config files with compilers If a compiler entry is already defined in a configuration file that the user does not know about, they may be confused when that compiler is not added by "spack compiler find". This commit adds a message at the end of "spack compiler find" to inform the user of the locations of all config files where compilers are defined.
2017-03-20Fix issue with config.guess patching when the overwritten config.guess did ↵Matthew LeGendre1-9/+5
not have write permissions. (#3494)
2017-03-18Use byte-encoded UTF-8 for sourced environment in Python 2 (#3489)Todd Gamblin3-9/+52
- Fixes recurring errors on develop with unicode commit characters. - still Python3-proof: python3 will use str instead of bytestrings.
2017-03-17Don't use @system in packages.yaml (#3472)Adam J. Stewart3-73/+19
2017-03-16Detect when OS updates affect compiler selection (#3410)scheibelp3-21/+77
Fixes #1476 Concretization uses compilers defined in config files and if those are not available defaults to searching typical paths where the detected operating system would have a compiler. If there is an OS update, the detected OS can change; in this case all compilers defined in the config files would no longer match (because they would be associated with the previous OS version). The error message in this case was too vague. This commit adds logic for detecting when it is likely that the OS has been updated (in particular when that affects compiler concretization) and improves the information provided to the user in the error message.
2017-03-16Dont propagate flags between different compilers (#3379)scheibelp2-41/+28
* Dont propagate flags between different compilers Fixes #2786 Previously when a spec had no parents with an equivalent compiler, Spack would default to adding the compiler flags associated with the root of the DAG. This eliminates that default. * added test for compiler flag propagation * simplify compiler flag propagation logic
2017-03-16Dont auto-init compiler conf for 'compiler find' (#3439)scheibelp2-8/+9
Fixes #3428 Users can run 'spack compiler find' to automatically initialize their compilers.yaml configuration file. It also turns out that Spack will implicitly initialize the compilers configuration file as part of detecting compilers if none are found (so if a user were to attempt to concretize a spec without running 'spack compiler find' it would not fail). However, in this case Spack was overlooking its own implicit initialization of the config files and would report that no new compilers were found. This commit removes implicit initialization when the user calls 'spack compiler find'. This did not surface until #2999 because the 'spack compiler' command defaulted to using a scope 'user/platform' that was not accounted for in get_compiler_config (where the implicit initialization logic predates the addition of this new scope); #2999 removed the scope specification when checking through config files, leading to the implicit initialization.
2017-03-14Consistent docs and usage of env mod methods (#3351)Adam J. Stewart2-81/+61
2017-03-14fix automatic mixing of clang with gfortran 6.3.0 on macOS (#3427)Denis Davydov2-14/+5
* fix automatic mixing of clang with gfortran 6.3.0 on macOS * automatically mix any gfortran with any clang on macOS * adjust the unit test
2017-03-14test/file_cache.py: ported to pytest (#3429)Massimiliano Culpo1-44/+36
2017-03-13Fix from_sourcing_files(): decode json input as utf-8 (#3433)Todd Gamblin1-2/+4
2017-03-11Correct inconsistency in comment (#3414)Erik Schnetter1-1/+1
2017-03-11Make multimethods work with inheritance. (#3411)Todd Gamblin2-3/+17
Previously, this would fail with a NoSuchMethodError: class Package(object): # this is the default implementation def some_method(self): ... class Foo(Package): @when('platform=cray') def some_method(self): ... @when('platform=linux') def some_method(self): ... This fixes the implementation of `@when` so that the superclass method will be invoked when no subclass method matches. Adds tests to ensure this works, as well.
2017-03-10Features/compiler config consistency (#2999)scheibelp6-70/+181
* default scope for config command is made consistent with cmd/__init__ default * dont specify a scope when looking for compilers with a matching spec (since compiler concretization is scope-independent) * config edit should default to platform-specific file only for compilers * when duplicate compiler specs are detected, the exception raised now points the user to the files where the duplicates appear * updated error message to emphasize that a spec is duplicated (since multiple specs can reference the same compiler) * 'spack compilers' is now also broken down into sections by os and target * Added tests for new compiler methods
2017-03-09created elf virtual package and updated dependent packages (#3317)Gregory Lee1-0/+13
* created elf virtual package and updated dependent packages * added `hide_files` context manager to handle moving files.
2017-03-07package.py: packages dump build dependencies in prefix (#3373)Massimiliano Culpo1-1/+3
Modifications: - `dump_packages` copies build dependencies into `$prefix/.spack`, as well as the link/run dependencies that we already copied there. - fake installs copy dependency packages into `$prefix/.spack` as well
2017-03-07test/environment.py: ported to pytest, added a test on separators (#3375)Massimiliano Culpo1-171/+235
2017-03-04Allow find_libraries to accept lists or strings (#3363)Adam J. Stewart2-14/+16
* Allow find_libraries to accept lists or strings * Convert one more example from list to string
2017-03-03Fix error raised for multiple virtual packages. (#3306)Joschka Lingemann1-1/+2
2017-03-02New interface for passing build information among specs (#1875)Massimiliano Culpo5-40/+285
- Added a new interface for Specs to pass build information - Calls forwarded from Spec to Package are now explicit - Added descriptor within Spec to manage forwarding - Added state in Spec to maintain query information - Modified a few packages (the one involved in spack install pexsi) to showcase changes - This uses an object wrapper to `spec` to implement the `libs` sub-calls. - wrapper is returned from `__getitem__` only if spec is concrete - allows packagers to access build information easily
2017-03-02test/packages: fixed test suite (#3236)Massimiliano Culpo1-106/+99
It seems the tests in `packages.py` were running just because we had a specific order of execution. This should fix the problem, and make the test_suite more resilient to running order.
2017-02-21Fixing issue wheh overwriting build_args in PythonPackage (#3200)Javier1-1/+1
2017-02-17Fix readline support in `spack python` (#3132)Adam J. Stewart3-0/+87
* Fix readline support in `spack python` * Add documentation on `spack python` * Add unit tests for `spack python`
2017-02-17Set default module type based on modules.yaml (#3173)Adam J. Stewart1-2/+4
2017-02-17More consistent yes/no prompts (#3174)Adam J. Stewart6-54/+51
* More consistent yes/no prompts * Add ==> prefix to yes/no and number prompts
2017-02-17On uninstall, change shortcut flag for --dependents to -R, as per Spack ↵Elizabeth Fischer1-1/+1
convention. (-r = --dependencies, -R = --dependents). (#1917)
2017-02-17Fix missing space in error message output (#3169)Adam J. Stewart1-2/+2
2017-02-17py-git2: New Package (#1971)Elizabeth Fischer1-1/+2
2017-02-17Fix typos in External Packages documentation (#3168)Adam J. Stewart1-2/+2
2017-02-10fix: don't call setup_environment when not needed (#3060)Massimiliano Culpo3-2/+16
* Don't call setup_environment when not needed. fixes #3059 * setup_environment and modules: added unit tests
2017-02-10Make distro more robust to unreadable files (#3110)becker331-7/+12
* Make distro more robust to unreadable files. Will upstream * Comment for clarify
2017-02-09make svn fetch quiet (#3070)Denis Davydov1-1/+1
2017-02-08Revert "db and concretization of packages modified after installation: fixes ↵Todd Gamblin1-1/+0
#2911" (#3078)
2017-02-07db and concretization of modified packages: fixes #2911 (#2920)Massimiliano Culpo1-0/+1
2017-02-03Fix spec hash printing (#2941)becker336-18/+18
- Fix format printing to match command line for hashes and full name formats - Update spack graph to use new format - Changed format string signifier for hashes from `$#` to `$/`
2017-02-03Make module autoload warnings configurable (#2763)George Hartzell1-3/+24
Modules generated by the module creation machinery currently print out a notice that warnts the user that things are being autoloaded. In some situations those warnings are problematic. See #2754 for discussion. This is a first cut at optionally disabling the warning messages: - adds a helper tothe EnvModule base class that encapsulates the config file variable; - adds a method to the base class that provides a default (empty) code fragment for generating a warning message; - passes the warning fragment into the bit that formats the autoload string; - adds specialized autload_warner() methods in the tcl and lmod subclasses;; and finally - touches up the autoload_format strings in the specialized classes.
2017-02-03Blacklist implicit packages for modulefile generation (#2603)George Hartzell2-1/+21
Add the ability to the modules generation process to blacklist packages that were installed implicitly. One can still whitelist modules that were installed implicitly. This changes adds a `blacklist_implicts` boolean as a peer to the `whitelist` and `blacklist` arrays, e.g.: ``` modules: enable:: - lmod lmod: whitelist: - 'lua' - 'py-setuptools' blacklist: - '%gcc@4.8.3' blacklist_implicits: True ``` It adds a small helper in `spec.py` and then touches up the package filtering code in `modules.py`.