summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2018-03-23Add the ability to separate architecture in path scheme (#4329)Mario Melara3-6/+67
* Add format to separate target and os for path spec format can now handle separations of target and os for setting up the path. * Added ${PLATFORM} et al to spec.format() ${PLATFORM}, ${OS}, ${TARGET} * Update tests Updated tests and got rid of unnecessary code. * Also update documentation to reflect this new ability. * Add default path scheme to config.yaml Added default path scheme to config.yaml. Users can overwrite this section if they want.
2018-03-22Speedup the default 'libs' property search ... (#7553)Veselin Dobrev1-24/+21
* Speedup the default 'libs' property search - important for external packages. * As advised by @alalazo, use tuples instead of lists inside _libs_default_handler.
2018-03-22Fix error raising during install with Python 3 (#6894)Adam J. Stewart1-2/+8
* Fix error raising during install with Python 3 * Fix env_path too
2018-03-22Record installation date and time in DB (#7334)Massimiliano Culpo6-51/+262
* Added installation date and time to the database Information on the date and time of installation of a spec is recorded into the database. The information is retained on reindexing. * Expose the possibility to query for installation date The DB can now be queried for specs that have been installed in a given time window. This query possibility is exposed to command line via two new options of the `find` command. * Extended docstring for Database._add * Use timestamps since the epoch instead of formatted date in the DB * Allow 'pretty date' formats from command line * Substituted kwargs with explicit arguments * Simplified regex for pretty date strings. Added unit tests.
2018-03-22Recover the old patching order for packages. (#7558)Massimiliano Culpo1-1/+1
fixes #7543 This is very likely an hot-fix, while a more permanent solution is needed. See this comment for more insight: https://github.com/spack/spack/pull/7193#discussion_r176448831 on the problem.
2018-03-21Propagate architecture information during concretization (#7412)becker332-17/+57
This updates architecture concretization to * Search for the nearest parent in the DAG for architecture information rather than defaulting to the root of the DAG * Propagate architecture settings transitively, such that if for example the target is set at the root of the dag it will set the same target on indirect dependencies (assuming no intermediate dependency specifies a separate target). Previously this occurred in general but under some conditions did not, for example if an intermediate dependency specified some subset of architecture properties.
2018-03-21Enable creation of mirrors for systems with different compilers (#5153)scheibelp3-13/+50
* Create mirror for system with different compilers Spack concretizes the spec provided by the user in "spack mirror create" to ensure downloading the right dependencies. Under normal circumstances concretization requires that the chosen compiler exists on the system, but this is not required when creating download mirrors for other systems, so this requirement is removed in that case. * Add test for disabling compiler existence check * Update compiler existence checking logic * improve test for disabling compiler existence check
2018-03-21Generate spack docset for Dash. (#6981)健美猫1-1/+7
2018-03-21make py-setuptools a run-time-only dep for py-basemap and patch pytho… (#6989)Gregory Lee1-2/+8
* make py-setuptools a run-time-only dep for py-basemap and patch python package to only apply setuptools flag for build deps * py-qtconsole does not require setuptools
2018-03-21Support MD5 with a warning when OpenSSL is compiled in FIPS mode (#7531)scheibelp1-3/+48
* This allows Spack to work with MD5 hashes on machines with openssl in FIPS mode. * We are still using MD5 for validation in many places, and a later PR will replace all uses of MD5 with SHA256. * This is a quick fix until that happens.
2018-03-21patch config.guess for ARM (aarch64) (#7427)Gregory Lee1-3/+3
2018-03-20spec: support "full" hashesPeter Scheibel2-3/+34
This hash includes the content of the package itself as well as the DAG for the package.
2018-03-20package: add a content_hash method for packagesPeter Scheibel2-0/+56
This calculates a hash which depends on the complete content of the package including sources and the associated `package.py` file.
2018-03-20package: ensure patches are applied in-orderPeter Scheibel1-1/+11
This helps to ensure that patches are applied consistently and will also be used as the source for the patch part of full package hashes.
2018-03-20package_hash: add code to generate a hash for a package filePeter Scheibel2-0/+235
This will be included in the full hash of packages.
2018-03-20PackageBase: add a list of metadata attributesPeter Scheibel1-0/+4
These attributes are ignored when doing a content hash of a package.
2018-03-20fetch_strategy: add a source_id to fetchesPeter Scheibel1-0/+46
This will be included in the full hash of packages.
2018-03-12Fix bugs in `spack dependencies` command (#7435)Todd Gamblin3-20/+35
- transitive dependencies were not being handled correctly - restructure code to do recursion and mark visited packages properly - add `-V` option to *not* expand virtuals in spack dependencies
2018-03-12Constructing a SpecBuildInterface from another gives no inconsistent MRO (#7457)Massimiliano Culpo2-1/+34
fixes #7239
2018-03-09spack buildcache: allow unsigned tarballs (#7440)Patrick Gartung4-67/+49
This re-adds the option to create and install unsigned tarballs, now with the -u option (--unsigned) rather than the -y option. This also changes the "keys" command, replacing the -y/--yes-to-all option with the -t/--trust option (which has the same effect but is more-clearly named).
2018-03-09resource: convert symlink to real path (#7417)Levi Baber1-1/+1
Fixes #7130 shutil.move expects a source path like "/x/y/" to be a directory and fails if "/x/y" is a symlink. This invokes realpath on the source path to avoid the issue.
2018-03-09replace dash with underscore in os name/version (#7381)scheibelp1-2/+2
Fixes #7356 In some cases OperatingSystem (e.g. LinuxDistro) was getting instantiated with a version that contains dashes. This breaks because the concretizer later converts this value to a string and re-parses it, and the '-' character is used to separate architecture components. This adds a guard in the initializer to convert '-' to '_'.
2018-03-07Changed CudaPackage base class to PackageBase (#7416)Massimiliano Culpo1-2/+2
CudaPackage is always used as a mixin. As such it should inherit from PackageBase, and not Package.
2018-03-06Improved binary relocation: error checking and reporting (#7314)Patrick Gartung5-149/+352
Fixes #7237 Fixes #6404 Fixes #6418 Fixes #6369 Identify when binary relocation fails to remove all instances of the source prefix (and report an error to the user unless they specify -a to allow the old root to appear). This check occurs at two stages: during "bincache create" all instances of the root are replaced with a special placeholder string ("@@@@..."), and a failure occurs if the root is detected at this point; when the binary package is extracted there is a second check. This addresses #7237 and #6418. This is intended to be compatible with previously-created binary packages. This also adds: * Better error messages for "spack install --use-cache" (#6404) * Faster relocation on Mac OS (using a single call to install_name_tool for all files rather than a call for each file) * Clean up when "buildcache create" fails (addresses #6369) * Explicit error message when the spack instance extracting the binary package uses a different install layout than the spack instance that created the binary package (since this is currently not supported) * Remove the option to create unsigned binary packages with -y
2018-03-05Fix pkgconf on Crays (#7171)Mario Melara1-0/+3
This updates Cray.setup_platform_environment to use cray-specific pkgconfig paths so that all providers of 'pkgconfig' have access to them (previously the 'pkg-config' provider had this but the 'pkgconf' provider did not).
2018-03-05only filter compiler paths for binaries provided by the compiler (#7369)scheibelp1-4/+9
2018-02-28[spack/spec.py] raise a query failure error if a property query returns None ↵Veselin Dobrev1-23/+41
(#7277) * [SPACK/spec.py] When a query through ForwardQueryToPackage returns 'None', treat that as query failure and raise RuntimeError with suitable message. This overrides the current behavior to raise an AttributeError which is now triggered only when no suitable query property is found and there is no default handler. * [spack/spec.py] Fix style. * [SPACK/spec.py] In case of query failure, i.e. property returning 'None', raise AttributeError instead of RuntimeError in order to pass the unit test. Also, small update in the logic distinguishing query failure and lack of relevant property/attribute handling.
2018-02-27Fix class name in build system tutorials (#7346)Mario Melara2-2/+2
AutoTools -> Autotools
2018-02-27Docker-Based Developer Resource (#5871)Omar Padron3-0/+62
* add docker-based development resources * incorporate feedback from @ax3l * fix docs/improve ssh handling * experiment with custom prompt * add arch/fix missing core_compilers key * add centos/minor tweaks * make prompt experiment optional * workaround problem with latest fedora docker image * add docker documentation page to index toc * try another documentation fix * switch arch linux base to base/archlinux * update the git urls in the Dockerfiles * add opensuse * switch CUSTOM_PROMPT variable to simply "PROMPT"
2018-02-23Pin Sphinx version to v1.7.0 (#7337)Massimiliano Culpo1-1/+1
Later versions of Sphinx fail due to possible clashes with our custom argparse.
2018-02-23Make fix_darwin_install_name python3-compatible (#7329)scheibelp1-10/+6
This updates the fix_darwin_install_name function to use the Spack Executable object to run install_name_tool, which ensures that process output is formatted as a 'str' for python2 and python3. Originally fix_darwin_install_name was invoking subprocess.Popen directly.
2018-02-15Use abspath to avoid permission errors(#7241)Pramod S Kumbhar1-0/+1
Fixes #5189 When working with non-normalized paths containing ".." on some file systems, Spack was found to encounter a permission error when writing to the path. This normalizes a path written by the intel-parallel-studio package and also normalizes all paths written by the license install hook (for all packages) to avoid this issue for intel-parallel-studio.
2018-02-13find has been changed to accept glob expressionsalalazo7-44/+79
Following the discussion with Todd and Adam, find has been modified to accept glob expressions. This should not affect performance as every glob implementation I inspected has 3 cases (no wildcard, wildcard but no directories involved, wildcard and directories involved) and uses fnmatch underneath. Mixins have been changed to do by default a non-recursive search (but a recursive search can still be triggered using the recursive keyword).
2018-02-13filter_compiler_path: added the possibility to narrow search pathalalazo1-10/+33
Following a comment from Todd, the search path for the files listed in `filter_compiler_wrappers` can now be narrowed. Anyhow, the function implementation still makes use of `find`, the rationale being that we have already seen packages that install artifacts in e.g. architecture dependent folders. The possibility to have a relative search path might be a good compromise between the previous approach and the one suggested in the review. Also: 'ignore_absent' and 'backup' keyword arguments can be optionally forwarded to `filter_file`.
2018-02-13mixins: moved debug logs to 'filter_file'. Renamed shadowed variable name.alalazo2-11/+16
Following comments from Todd: - the call to tty.debug has been moved deeper, to log the filtering of each file - the shadowing on the name "kwargs" is avoided
2018-02-13mixins: flush method dictionaries after registration.alalazo1-0/+6
2018-02-13mixins: implemented declarative syntaxalalazo4-54/+144
Implemented a declarative syntax for the additional behavior that can get attached to classes. Implemented a function to filter compiler wrappers that uses the mechanism above.
2018-02-13Improved naming of properties and classes (per Denis comments).alalazo2-9/+11
2018-02-13Added mixins to modularize common behaviors across build-systems.alalazo3-0/+90
Modifications: * added a mixin to filter compiler wrappers from files * modified hdf5, openmpi, mpich, mvapich2 to use it
2018-02-12Rework command reference in docs, add `spack commands` commandTodd Gamblin7-20/+459
- command reference now includes usage for all Spack commands as output by `spack help`. Each command usage links to any related section in the docs. - added `spack commands` command which can list command names, subcommands, and generate RST docs for commands. - added `llnl.util.argparsewriter`, which analyzes an argparse parser and calls hooks for description, usage, options, and subcommands
2018-02-12Fix issues with `spack [cmd] -h` help outputTodd Gamblin2-13/+40
- Shorten Spack command usage for short options. Short options are now shown as [-abc] instead of as [-a] [-b] [-c] - fix bug that mixed long and short options for top-level `spack help`
2018-02-12Fix section labels for some commands in the docsTodd Gamblin2-7/+7
2018-02-12make defaults in command help look like standard argparseTodd Gamblin5-7/+7
2018-02-12Add help for buildcache, reorganize command help categoriesTodd Gamblin5-10/+15
- Add proper help for `spack buildcache` subcommands - Reorganize the help categories of Spack commands so that buildcache is in packaging and diy and setup are now in build.
2018-02-12Clean up help metavar for --scope configuration optionsTodd Gamblin6-14/+32
- previously commands with this argument showed a long list of choices that were platform specific. - use a better metavar: {defaults,system,site,user}[/PLATFORM]
2018-02-07Change location of public key in basic tutorial (#7192)v0.11.2becker331-5/+5
* Change location of public key in basic tutorial * Change tuturial version number to 11.2
2018-02-02Fix ignore-conflicts for extensions in views (#7167)healther3-9/+28
Fixes #7159 When activating extensions in external views, the --ignore-conflicts option was being ignored. In this particular issue the conflict was for the duplicate __init__ file for multiple python packages in the same namespace, but in general any conflict for extensions would cause an error whether or not --ignore-conflicts was set. This also renames the 'force' option of do_activate to 'with_dependencies' and updates views to call do_activate with this set to False (since it traverses the dependency dag anyway). This isn't strictly required, it just avoids redundant calls.
2018-02-01Update tutorial on advanced packaging (#7144)scheibelp1-278/+227
This reorganizes most sections and rewords a significant portion of the content (including all introductions) but keeps all the examples. * Remove section 'What happens at subscript time' from tutorial: it is too detailed for a tutorial * Move the 'Extra query parameters' and 'Attach attributes to other packages' sections into a separate grouping 'Other packaging topics' * move the 'Set variables at build time yourself' section after 'Set environment variables in dependents' section since the latter is more motivating * start the 'set environment variables at build-time for yourself' section with qt as an example * renamed section 'specs build interface' to 'retrieving library information' and updated section introduction * renamed section 'a motivating example' to 'accessing library dependencies'; split out the material which deals with implementing .libs for netlib-lapack into a separate section called 'providing libraries to dependents'. consolidated in material from the section 'single package providing multiple virtual specs' since netlib-lapack is an example of this (this removes the material about intel-parallel studio)
2018-01-31Improve log parsing performance (#7093)Todd Gamblin6-96/+408
* Allow dashes in command names and fix command name handling - Command should allow dashes in their names like the reest of spack, e.g. `spack log-parse` - It might be too late for `spack build-cache` (since it is already called `spack buildcache`), but we should try a bit to avoid inconsistencies in naming conventions - The code was inconsistent about where commands should be called by their python module name (e.g. `log_parse`) and where the actual command name should be used (e.g. `log-parse`). - This made it hard to make a command with a dash in the name, and it made `SpackCommand` fail to recognize commands with dashes. - The code now uses the user-facing name with dashes for function parameters, then converts that the module name when needed. * Improve performance of log parsing - A number of regular expressions from ctest_log_parser have really poor performance, most due to untethered expressions with * or + (i.e., they don't start with ^, so the repetition has to be checked for every position in the string with Python's backtracking regex implementation) - I can't verify that CTest's regexes work with an added ^, so I don't really want to touch them. I tried adding this and found that it caused some tests to break. - Instead of using only "efficient" regular expressions, Added a prefilter() class that allows the parser to quickly check a precondition before evaluating any of the expensive regexes. - Preconditions do things like check whether the string contains "error" or "warning" (linear time things) before evaluating regexes that would require them. It's sad that Python doesn't use Thompson string matching (see https://swtch.com/~rsc/regexp/regexp1.html) - Even with Python's slow implementation, this makes the parser ~200x faster on the input we tried it on. * Add `spack log-parse` command and improve the display of parsed logs - Add better coloring and line wrapping to the log parse output. This makes nasty build output look better with the line numbers. - `spack log-parse` allows the log parsing logic used at the end of builds to be executed on arbitrary files, which is handy even outside of spack. - Also provides a profile option -- we can profile arbitrary files and show which regular expressions in the magic CTest parser take the most time. * Parallelize log parsing - Log parsing now uses multiple threads for long logs - Lines from logs are divided into chnks and farmed out to <ncpus> - Add -j option to `spack log-parse`
2018-01-29Mark slow unit tests (#6994)Massimiliano Culpo14-71/+136
* Marking database tests as slow * Marking url command tests as slow * Marking every test that uses database as slow * Marking tests that import files as slow * Marking gpg tests as slow * Marking all versions and one list tests as slow * Added more markers to unit tests + cli option to skip slow tests Following a discussion with Axel, the generic 'slowtest' marker has been split into 'db', 'network' and 'maybeslow'. A brief description of the meaning of each marker has been added to pytest.ini. A command line option to run only fast tests has been added to 'spack test' * Don't use classes to group tests together Reverted grouping tests under a class, as required in the review * Minor style changes