summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2017-05-02Add link to spack view docs in command index (#4082)Adam J. Stewart1-3/+4
2017-05-01Multi-valued variants (#2386)Massimiliano Culpo16-187/+1703
Modifications: - added support for multi-valued variants - refactored code related to variants into variant.py - added new generic features to AutotoolsPackage that leverage multi-valued variants - modified openmpi to use new features - added unit tests for the new semantics
2017-05-01Skip fetch tests for tools that are not installed. (#4059)Todd Gamblin4-9/+39
This allows people on systems that don't have all the fetchers to still run Spack tests. Mark tests that require git, subversion, or mercurial to be skipped if they're not installed.
2017-05-01url_parse: ported to pytest (#3430)Massimiliano Culpo1-693/+391
2017-05-01namespace_trie: ported to pytest (#4060)Massimiliano Culpo1-59/+63
2017-05-01Add a WafPackage base class (#3975)Adam J. Stewart7-35/+201
* Add a WafPackage base class * Correct comment in docstring * Be more specific about the Python versions supported
2017-05-01url_substitution: ported to pytest (#4032)Massimiliano Culpo1-54/+34
2017-04-30disable rpaths on Darwin when arg=-r mode=ccld (#3930)Denis Davydov1-1/+1
This fixes build of Ipopt package.
2017-04-30Updated cc wrapper: switch from ld to vcheck if version is requested. (#2501)Sergey Kosukhin1-1/+1
2017-04-30Don't add system paths to PATH (#3910)Adam J. Stewart3-84/+72
* Filter all system paths introduced by dependencies from PATH * Make sure path filtering works *even* for trailing slashes * Revert some of the changes to `filter_system_paths` * Yes, `bin64` is a real thing (sigh) * add tests: /usr, /usr/, /usr/local/../bin, etc. * Convert from rST to Google-style docstrings
2017-04-30spack_yaml: ported to pytest (#4033)Massimiliano Culpo1-67/+69
2017-04-30file_list: ported to pytest (#4054)Massimiliano Culpo1-180/+158
2017-04-29fetch: do full clone of git submodules (fix #3956) (#3958)Christoph Junghans1-13/+4
The required hash of a submodule might point to the non-HEAD commit of the current main branch and hence would lead to a "no such remote ref" at checkout in a shallow submodule.
2017-04-29Python command, libraries, and headers (#3367)Adam J. Stewart6-189/+614
## Motivation Python installations are both important and unfortunately inconsistent. Depending on the Python version, OS, and the strength of the Earth's magnetic field when it was installed, the name of the Python executable, directory containing its libraries, library names, and the directory containing its headers can vary drastically. I originally got into this mess with #3274, where I discovered that Boost could not be built with Python 3 because the executable is called `python3` and we were telling it to use `python`. I got deeper into this mess when I started hacking on #3140, where I discovered just how difficult it is to find the location and name of the Python libraries and headers. Currently, half of the packages that depend on Python and need to know this information jump through hoops to determine the correct information. The other half are hard-coded to use `python`, `spec['python'].prefix.lib`, and `spec['python'].prefix.include`. Obviously, none of these packages would work for Python 3, and there's no reason to duplicate the effort. The Python package itself should contain all of the information necessary to use it properly. This is in line with the recent work by @alalazo and @davydden with respect to `spec['blas'].libs` and friends. ## Prefix For most packages in Spack, we assume that the installation directory is `spec['python'].prefix`. This generally works for anything installed with Spack, but gets complicated when we include external packages. Python is a commonly used external package (it needs to be installed just to run Spack). If it was installed with Homebrew, `which python` would return `/usr/local/bin/python`, and most users would erroneously assume that `/usr/local` is the installation directory. If you peruse through #2173, you'll immediately see why this is not the case. Homebrew actually installs Python in `/usr/local/Cellar/python/2.7.12_2` and symlinks the executable to `/usr/local/bin/python`. `PYTHONHOME` (and presumably most things that need to know where Python is installed) needs to be set to the actual installation directory, not `/usr/local`. Normally I would say, "sounds like user error, make sure to use the real installation directory in your `packages.yaml`". But I think we can make a special case for Python. That's what we decided in #2173 anyway. If we change our minds, I would be more than happy to simplify things. To solve this problem, I created a `spec['python'].home` attribute that works the same way as `spec['python'].prefix` but queries Python to figure out where it was actually installed. @tgamblin Is there any way to overwrite `spec['python'].prefix`? I think it's currently immutable. ## Command In general, Python 2 comes with both `python` and `python2` commands, while Python 3 only comes with a `python3` command. But this is up to the OS developers. For example, `/usr/bin/python` on Gentoo is actually Python 3. Worse yet, if someone is using an externally installed Python, all 3 commands may exist in the same directory! Here's what I'm thinking: If the spec is for Python 3, try searching for the `python3` command. If the spec is for Python 2, try searching for the `python2` command. If neither are found, try searching for the `python` command. ## Libraries Spack installs Python libraries in `spec['python'].prefix.lib`. Except on openSUSE 13, where it installs to `spec['python'].prefix.lib64` (see #2295 and #2253). On my CentOS 6 machine, the Python libraries are installed in `/usr/lib64`. Both need to work. The libraries themselves change name depending on OS and Python version. For Python 2.7 on macOS, I'm seeing: ``` lib/libpython2.7.dylib ``` For Python 3.6 on CentOS 6, I'm seeing: ``` lib/libpython3.so lib/libpython3.6m.so.1.0 lib/libpython3.6m.so -> lib/libpython3.6m.so.1.0 ``` Notice the `m` after the version number. Yeah, that's a thing. ## Headers In Python 2.7, I'm seeing: ``` include/python2.7/pyconfig.h ``` In Python 3.6, I'm seeing: ``` include/python3.6m/pyconfig.h ``` It looks like all Python 3 installations have this `m`. Tested with Python 3.2 and 3.6 on macOS and CentOS 6 Spack has really nice support for libraries (`find_libraries` and `LibraryList`), but nothing for headers. Fixed.
2017-04-28Add tests to MakefilePackage (#4039)Adam J. Stewart1-0/+13
2017-04-28No compiler found: fixed error message (#4034)Massimiliano Culpo1-3/+4
When a compiler was not found a stacktrace was displayed to user because there were three arguments to be substituted in a string with only two substitutions to be done.
2017-04-27Reindex checks install for non-external packages (#4027)scheibelp1-7/+12
Fixes #4026 #1167 updated Database.reindex to keep old installation records to support external packages. However, when a user manually removes a prefix and reindexes this kept the records so the packages were still installed according to "spack find" etc. This adds a check for non-external packages to ensure they are properly installed according to the directory layout.
2017-04-27Prevent spack test flake8 from making changes (#4023)Adam J. Stewart1-2/+2
2017-04-27Remove 'release' suffix from package name (#4014)Adam J. Stewart2-0/+5
2017-04-27Don't print successfully uninstalled twice (#4019)Adam J. Stewart1-1/+0
2017-04-27Fix Python3 issue with sbang checking; add tests. (#4017)Todd Gamblin2-49/+76
2017-04-27Convert rest docstrings to Google docstrings. (#3994)Todd Gamblin18-284/+360
- Sometimes you need something mindless to do. - Sometimes it can be helpful, as well.
2017-04-27pattern: ported to pytest (#4015)Massimiliano Culpo1-57/+60
2017-04-26Fix alignment of versions and urls in spack checksum (#4003)Adam J. Stewart1-1/+1
2017-04-26link_tree: ported to pytest (#4008)Massimiliano Culpo1-89/+98
2017-04-26Add __format__ support to version for fancy formatting. (#3996)Todd Gamblin1-0/+3
- add Version.__format__ to support new-style formatting. - Python3 doesn't handle this well -- it delegates to object.__format__(), which raises an error for fancy format strings. - not sure why it doesn't call str(self).__format__ instead, but that's hwo things are.
2017-04-25Add API Docs for lib/spack/llnl (#3982)Adam J. Stewart10-101/+130
* Add API Docs for lib/spack/llnl * Clean up after previous builds * Better fix for purging API docs
2017-04-25Fix bug with '# noqa' filtering (#3993)Adam J. Stewart1-9/+4
2017-04-25spack flake8 should exempt line-wrapped directives (#3990)Todd Gamblin1-8/+8
- Omit final paren from regular expressions in cmd/flake8.py - Allows long directives to be exempted even if they are wrapped.
2017-04-25hooks take spec as an argument (instead of pkg) (#3967)Massimiliano Culpo6-23/+27
2017-04-25Properly ignore flake8 F811 redefinition errors (#3932)Adam J. Stewart2-33/+153
* Properly ignore flake8 F811 redefinition errors * Add unit tests for flake8 command * Allow spack flake8 to work on systems with older git * Skip flake8 unit tests for Python 2.6 and 3.3
2017-04-24Rename tutorial_sc16.rst to tutorial.rst (#3974)Todd Gamblin5-4/+4
- make the name more generic for posterity.
2017-04-23Use six.moves.input instead of raw_input (#3961)Adam J. Stewart2-3/+4
* Use six.moves.input instead of raw_input * Remove comment mentioning raw_input
2017-04-22External packages are now registered in the DB (#1167)Massimiliano Culpo15-57/+230
* treats correctly a change from `explicit=False` to `explicit=True` in an external package DB entry. * added unit tests * fixed issues raised by @tgamblin . In particular the PR is no more hash-changing for packages that are not external. * added a test to check correctness of a spec/yaml round-trip for things that involve an external * Don't find external module path at each step of concretization * it's not necessary.. The paths are retrieved at the end of concretizaion * Don't find replacements for external packages. * Test root of the DAG if external * No reason not to test if the root of the DAG is external when external packages are now first class citizens! * Create `external` property for Spec (for external_path and external_module) * Allow users to specify external package paths relative to spack * Canonicalize external package paths so that users may specify their locations relative to spack's directory. * Update tests to use new external_path and external properly. * skip license hooks on external
2017-04-22Find more versions from GitHub (#3952)Adam J. Stewart1-1/+1
2017-04-22add Intel mpi package and add MPI wrappers to Intel parallel studio (#3905)Denis Davydov1-0/+1
* intel-mpi: add new package * fix hashes * fix typo * flake8 * add install * blank line * error * add bin64 * fix MPI wrappers in intel-parallel-studio * add missing I_MPI_FC * use shorter hashes
2017-04-21PythonPackage builds flat installs instead of egg directories.Todd Gamblin1-1/+18
- Spack doesn't need eggs -- it manages its own directories - Simplify install layout and reduce sys.path searches by installing all packages flat (eggs are deprecated for wheels, and this is also what wheels do). - We now supply the --single-version-externally-managed argument to `setup.py install` for setuptools packages and setuptools. - modify packages to only use setuptools args if setuptools is an immediate dependency - Remove setuptools from packages that do not need it. - Some packages use setuptools *only* when certain args (likeb 'develop' or 'bdist') are supplied to setup.py, and they specifically do not use setuptools for installation. - Spack never calls setup.py this way, so just removing the setuptools dependency works for these packages.
2017-04-21fetch git submodules recursively (#3779)Christoph Junghans2-5/+16
* fetch git submodules recursively This is useful if the submodules have submodules themselves. On the other hand doing a recursive update doesn't hurt if there is only one level. * fetch submodules with depth=1 as well (fix #2190) * use git submodule with depth only for git>=1.8.4
2017-04-21spack uninstall no longer requires a known package. (#3915)Todd Gamblin6-82/+102
- Spack install would previously fail if it could not load a package for the thing being uninstalled. - This reworks uninstall to handle cases where the package is no longer known, e.g.: a) the package has been renamed or is no longer in Spack b) the repository the package came from is no longer registered in repos.yaml
2017-04-21Fix ABI detection issues with macOS gcc. (#3854)Todd Gamblin4-7/+23
- gcc on macOS says it's version 4.2.1, but it's really clang, and it's actually the *same* clang as the system clang. - It also doesn't respond with a full path when called with --print-file-name=libstdc++.dylib, which is expected from gcc in abi.py. Instead, it gives a relative path and _gcc_compiler_compare doesn't understand what to do with it. This results in errors like: ``` lib/spack/spack/abi.py, line 71, in _gcc_get_libstdcxx_version libpath = os.readlink(output.strip()) OSError: [Errno 2] No such file or directory: 'libstdc++.dylib' ``` - This commit does two things: 1. Ignore any gcc that's actually clang in abi.py. We can probably do better than this, but it's not clear there is a need to, since we should handle the compiler as clang, not gcc. 2. Don't auto-detect any "gcc" that is actually clang anymore. Ignore it and expect people to use clang (which is the default macOS compiler anyway). Users can still add fake gccs to their compilers.yaml if they want, but it's discouraged.
2017-04-21Fix checksumming in Python3; add more fetch tests (#3941)Todd Gamblin7-59/+151
* Checksum code wasn't opening binary files as binary. - Fixes Python 3 issue where files are opened as unicode text by default, and decoding fails for binary blobs. * Simplify fetch test parametrization. * - add tests for URL fetching and checksumming. - fix coverage on interface functions in FetchStrategy superclass - add some extra crypto tests.
2017-04-21Fix HPL build, convert to MakefilePackage (#3777)Adam J. Stewart1-0/+49
* Fix HPL build, convert to MakefilePackage * Flake8 fix * Fix: spec -> self.spec * Properly query for system libraries * Update Intel-MKL as well * Recurse in system libs, fix MKL path, fixes lapack_libs
2017-04-20Add Napoleon extension to support Google docstrings (#3920)Adam J. Stewart1-0/+1
2017-04-20Revert "Override partial installs by default" (#3918)Todd Gamblin4-203/+7
* Revert "Override partial installs by default (#3530)" This reverts commit a65c37f15dff4b4d60784fd4fcc55874ce9d6d11.
2017-04-20test/python_version.py: ported to pytest (#3438)Massimiliano Culpo1-79/+85
2017-04-19Override partial installs by default (#3530)scheibelp4-7/+203
* Package install remove prior unfinished installs Depending on how spack is terminated in the middle of building a package it may leave a partially installed package in the install prefix. Originally Spack treated the package as being installed if the prefix was present, in which case the user would have to manually remove the installation prefix before restarting an install. This commit adds a more thorough check to ensure that a package is actually installed. If the installation prefix is present but Spack determines that the install did not complete, it removes the installation prefix and starts a new install; if the user has enabled --keep-prefix, then Spack reverts to its old behavior. * Added test for partial install handling * Added test for restoring DB * Style fixes * Restoring 2.6 compatibility * Relocated repair logic to separate function * If --keep-prefix is set, package installs will continue an install from an existing prefix if one is present * check metadata consistency when continuing partial install * Added --force option to make spack reinstall a package (and all dependencies) from scratch * Updated bash completion; removed '-f' shorthand for '--force' for install command * dont use multiple write modes for completion file
2017-04-19Set proper deptypes for certain build systems (#3866)Adam J. Stewart3-4/+10
* Set proper deptypes for certain build systems * Add depends_on to namespace
2017-04-19Rework Spack's Mercurial support (#3834)Adam J. Stewart4-8/+59
* Add tests to mercurial package * Add support for --insecure with mercurial fetching * Install man pages and tab-completion scripts * Add tests and latest version for all deps * Flake8 fix * Use certifi module to find CA certificate * Flake8 fix * Unset PYTHONPATH when running hg * svn_fetch should use to svn-test, not hg-test * Drop Python 3 support in Mercurial Python 3 support is a work in progress and isn't currently recommended: https://www.mercurial-scm.org/wiki/SupportedPythonVersions * Test both secure and insecure hg fetching * Test both secure and insecure git and svn fetching
2017-04-15Fix flake8 changed files detection (#3855)Adam J. Stewart1-1/+1
2017-04-15Allow users to set parallel jobs in config.yaml (#3812)Adam J. Stewart5-6/+30
* Allow users to set parallel jobs in config.yaml * Undo change from endash to emdash * Remove parallel config, rename jobs to build_jobs