summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2020-02-19Add extra version of py-isort (#14988)iarspider1-1/+8
* Add extra version of py-isort * Update package.py * Update package.py * Update var/spack/repos/builtin/packages/py-isort/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-isort/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-isort/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-isort/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-19Add extra version of py-dask; add new variant and dependencies (#14973)iarspider4-0/+84
* Add extra version of py-dask * Update package.py * Add extra dependencies for py-dask+distributed * Update package.py * Update var/spack/repos/builtin/packages/py-heapdict/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-heapdict/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-distributed/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-distributed/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update package.py * Update package.py * Update package.py * Update package.py * Update var/spack/repos/builtin/packages/py-distributed/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-distributed/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update package.py * Update var/spack/repos/builtin/packages/py-distributed/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update package.py * Flake-8 * Add patch step for py-distributed Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-19Add extra version of py-pyzmq (#15080)iarspider1-0/+1
2020-02-19Fix gcc 8.3 build on macOS 10.14 (#15089)Seth R. Johnson1-0/+6
2020-02-19Buildcache: join rpaths returned by get_existing_elf_rpaths with ':'. (#15086)Patrick Gartung1-1/+1
This reproduces the behavior expected by patchelf_is_relocatable test.
2020-02-19Distributed builds (#13100)Tamara Dahlgren100-963/+2954
Fixes #9394 Closes #13217. ## Background Spack provides the ability to enable/disable parallel builds through two options: package `parallel` and configuration `build_jobs`. This PR changes the algorithm to allow multiple, simultaneous processes to coordinate the installation of the same spec (and specs with overlapping dependencies.). The `parallel` (boolean) property sets the default for its package though the value can be overridden in the `install` method. Spack's current parallel builds are limited to build tools supporting `jobs` arguments (e.g., `Makefiles`). The number of jobs actually used is calculated as`min(config:build_jobs, # cores, 16)`, which can be overridden in the package or on the command line (i.e., `spack install -j <# jobs>`). This PR adds support for distributed (single- and multi-node) parallel builds. The goals of this work include improving the efficiency of installing packages with many dependencies and reducing the repetition associated with concurrent installations of (dependency) packages. ## Approach ### File System Locks Coordination between concurrent installs of overlapping packages to a Spack instance is accomplished through bottom-up dependency DAG processing and file system locks. The runs can be a combination of interactive and batch processes affecting the same file system. Exclusive prefix locks are required to install a package while shared prefix locks are required to check if the package is installed. Failures are communicated through a separate exclusive prefix failure lock, for concurrent processes, combined with a persistent store, for separate, related build processes. The resulting file contains the failing spec to facilitate manual debugging. ### Priority Queue Management of dependency builds changed from reliance on recursion to use of a priority queue where the priority of a spec is based on the number of its remaining uninstalled dependencies. Using a queue required a change to dependency build exception handling with the most visible issue being that the `install` method *must* install something in the prefix. Consequently, packages can no longer get away with an install method consisting of `pass`, for example. ## Caveats - This still only parallelizes a single-rooted build. Multi-rooted installs (e.g., for environments) are TBD in a future PR. Tasks: - [x] Adjust package lock timeout to correspond to value used in the demo - [x] Adjust database lock timeout to reduce contention on startup of concurrent `spack install <spec>` calls - [x] Replace (test) package's `install: pass` methods with file creation since post-install `sanity_check_prefix` will otherwise error out with `Install failed .. Nothing was installed!` - [x] Resolve remaining existing test failures - [x] Respond to alalazo's initial feedback - [x] Remove `bin/demo-locks.py` - [x] Add new tests to address new coverage issues - [x] Replace built-in package's `def install(..): pass` to "install" something (i.e., only `apple-libunwind`) - [x] Increase code coverage
2020-02-18Add latest 3.9.x releases QMCPACK (#15056)Nichols A. Romero1-0/+3
2020-02-18octave: enable 64-bit BLAS builds. (#15035)Kai Torben Ohlhus1-0/+6
* octave: enable 64-bit BLAS builds. Perform necessary actions [as described in the manual](https://octave.org/doc/v5.2.0/Compiling-Octave-with-64_002dbit-Indexing.html). * Update package.py
2020-02-18modern-wheel: add virtual destructor to BaseMultiParms class (#14899)noguchi-k2-0/+21
* modern-wheel: add virtual destructor * modern-wheel: add maintainers
2020-02-18ENH: add catch2 CMake install (#15008)Tyler Reddy2-16/+21
* ENH: add catch2 CMake install * add a variant allowing catch2 to be installed via CMake, which is useful for generating a .cmake config file for consumption by other projects * Catch2: Simplify Package - CMake install is also single-header for new releases - testing triggered by Spack's test mechanism - default to CMake build (better than simple copy, which is just for old releases to be installed) * Catch: Remove Variant We can control all installs with CMake to be quick and complete. Old versions prior to 1.7.0 will be manually installed, as the `make install` target is missing in those. Releases 1.7.0-1.9.3 do not expose control over test builds. * openPMD-api: Catch Lost single_header ... variant is gone :) Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
2020-02-18slate needs mercurial for build (#15055)eugeneswalker1-0/+1
2020-02-18[hepmc] Finish the HepMC3 spackage update (#15036)Hadrien G1-1/+13
* Finish the HepMC3 spackage update - Update CMake requirement per latest master - Account for Python variant, add python dependency if used - Account for ROOT I/O variant, add ROOT dependency if used * Please flake8
2020-02-18Update and simplify julia package (#14756)Glenn Johnson1-174/+104
* Update and simplify julia package The current Spack Julia package potentially installs a few julia packages, with the installation being controlled by variants. There are a couple of problems with this. First, Julia handles packages very differently from systems such as R and Python. Julia requires write access to the repository directories in order for user installs of packages to work. If spack installs julia packages then there will be a repository, DEPOT_PATH, in the installation directory. If spack is used on an individual basis this would work but would mean that package data is written to the spack installation directory after installation. If spack is used to provide packages for end users then user installs of julia packages will fail due to lack of write access to the repository in the installation directory. It seems best for spack to just install julia without any julia packages, and drop the configuration for those packages. Second, having spack install package as variants seems to be counter to how spack works for other extendable systems, like R and Python. Julia should be an extendable package in spack but it is not clear how to make that work. As pointed out above, installing user packages requires write access to the julia repositories, including the one in the install directory. Essentially, a user package installation will try to update metadata for *all* julia repositories. Furthermore, each of those repositories, assuming one per package with spack, would need to have the Project.toml files merged to present the package stack to julia. Again, it seems best for spack to just install julia itself and not try to install julia packages, at least at this time. A good discussion on this can be found at https://discourse.julialang.org/t/how-does-one-set-up-a-centralized-julia-installation/13922. This PR does the following: - adds versions 1.2.0 and 1.3.1 - removes variants that correspond to julia packages - changes python to build dependency as it seems to only be needed for LLVM - the new versions can use Python-3 - removes dependencies for packages - adds a conflict statement for Intel compiler, with comment - add a setup_build_environment method to find GCC libraries - make formatting consistent - adds JULIA_CPU_TARGET option to correspond with target to help with running julia image on hardware older than build host - added intel build options, for when they can be used - removed code for installing packages - removed code for julia config that was needed for packages Note that versions below 0.5.1 fail to build, with or without these changes. It is not clear why that is. * Update var/spack/repos/builtin/packages/julia/package.py Yes, need to use correct grammar even in the midst of numbers and symbols. Good catch! Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * More cleanup of Julia package This commit does more cleanup and sets more constraints. - Removed release-0.4 and release-0.5. I am not sure if those are actually useful but they are quite old and there are released versions from the same timeframe. - Remove the binutils variant. - Made cmake a build dependency for versions >= 1. - Added git as a dependency for @master. - Limit curl dependency to released versions. - Do not use external curl for master. When I checked, using the external version failed but the internal curl worked. - Versions <= 0.5.0 need an older version of openssl. - Set conflicts directive for cxx variant. - Added conflicts directive for needing +mkl with Intel compiler. - Removed configuration settings as these prevented julia from working properly in all cases that I looked at. * Fix flake8 error Remove 'import sys' that is no longer used. * More dependency tweaks This commit sets further version constraints on dependencies. It really looks like julia requires its internal dependencies more over time. - curl only up to 0.5.0 - openssl only up to 0.5.0 - override with system curl up to version 0.5.0 * Fix spec for curl certs Only depending on curl through 0.5.0. Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-18buildcache: Check for tar.bz2 and set tar.gz if not found (#15054)Patrick Gartung1-0/+4
* Check for tar.bz2 and set tar.gz if not found * Move check for tarfile after it is extracted
2020-02-18Add extra version of py-cryptography (#14975)iarspider1-3/+14
* Add extra version of py-cryptography * Update package.py * Update package.py * Update var/spack/repos/builtin/packages/py-cryptography/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-cryptography/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-cryptography/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update package.py * Flake-8 Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-18ROSS: v7.2.0 and branches to match (#15052)Elsa Gonsiorowski, PhD1-1/+5
2020-02-18Fix OpenBLAS 0.3.8 build on darwin (#15041)Seth R. Johnson2-0/+27
* Fix OpenBLAS 0.3.8 build on darwin See https://github.com/xianyi/OpenBLAS/issues/2431, patched in https://github.com/xianyi/OpenBLAS/issues/2431 . * Add references for patchfile
2020-02-18py-hpcbench: new package (#14915)Matthias Wolf1-0/+32
* py-hpcbench: new package * obey the flake8 * address comments, fix versions. * Update var/spack/repos/builtin/packages/py-hpcbench/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-18Add extra version of py-graphviz (#14981)iarspider1-1/+4
* Add extra version of py-graphviz * Update package.py * Update package.py * Update var/spack/repos/builtin/packages/py-graphviz/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-18npm: Add 6.13.7 (#15034)Michael Kuhn1-2/+3
2020-02-18Add extra version of py-nbconvert (#15044)iarspider1-0/+1
2020-02-18Add extra version of py-pathlib2 (#15048)iarspider1-0/+1
2020-02-18Add extra version of py-pbr (#15049)iarspider1-0/+1
2020-02-18Add extra version of py-pexpect (#15050)iarspider1-0/+2
* Add extra version of py-pexpect * Update package.py Update dependency version
2020-02-18Add extra version of py-ipywidgets (#14987)iarspider1-2/+10
* Add extra version of py-ipywidgets * Update var/spack/repos/builtin/packages/py-ipywidgets/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update package.py * Update var/spack/repos/builtin/packages/py-ipywidgets/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-ipywidgets/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-ipywidgets/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-ipywidgets/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-ipywidgets/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> * Update var/spack/repos/builtin/packages/py-ipywidgets/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-18Add extra version of py-defusedxml (#14972)iarspider1-0/+2
* Add extra version of py-defusedxml * Update package.py * Update var/spack/repos/builtin/packages/py-defusedxml/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-18Add extra version of py-logilab-common (#14997)iarspider1-2/+5
* Add extra version of py-logilab-common * Update package.py * Update var/spack/repos/builtin/packages/py-logilab-common/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-18Fix allowed python versions of py-tables (#14073)iarspider1-1/+3
* Fix allowed python versions of py-tables * Update package.py * Update package.py * I think I've finally got it * Update package.py * Update package.py * Update package.py
2020-02-18Add extra version of py-more-itertools (#15042)iarspider1-0/+1
2020-02-18Add extra version of py-oauthlib (#15045)iarspider1-0/+1
2020-02-18Add extra version of py-owslib (#15046)iarspider1-0/+1
2020-02-18skip gpg tests when no gpg executable (#14935)Greg Becker4-1/+38
* skip gpg tests when no gpg executable * flake
2020-02-18Update `davix` versions and add version restriction for ROOT (#15038)Seth R. Johnson2-1/+3
From ROOT cmake output: ``` -- Checking for module 'davix>=0.6.4' -- Found davix, version 0.6.8 CMake Warning at cmake/modules/SearchInstalledSoftware.cmake:960 (message): Davix versions 0.6.8 to 0.7.0 have a bug and do not work with ROOT, please upgrade to 0.7.1 or later. ```
2020-02-18an argument 'buf_size' of 'h5fget_file_image_c' should be intent(out). (#15011)h-murai1-0/+16
* an argument 'buf_size' of 'h5fget_file_image_c' should be intent(out). * correct format errors * some modifications based on the comments from the reviewer
2020-02-18pegtl: change to virtual destructor (#14939)noguchi-k2-1/+17
* pegtl: change to virtual destructor * pegtl: fix patch
2020-02-17py-cookiecutter: new package (#14911)Matthias Wolf1-0/+27
* py-cookiecutter: new package * address comments
2020-02-17node-js: Add 12.16.0 and 13.8.0 (#15024)Michael Kuhn1-1/+3
2020-02-17Fix CLHEP fetch URL and add new version (#15032)Seth R. Johnson1-1/+2
2020-02-17spack extensions prints list of extendable packages (#14473)Adam J. Stewart4-3/+23
* spack extensions prints list of extendable packages * Update tab completion scripts
2020-02-17Fix mercurial certificate finding, add latest version (#15026)Adam J. Stewart1-12/+13
2020-02-17Add new version of cairo (#14941)iarspider1-2/+5
* Add new version of cairo * Update var/spack/repos/builtin/packages/cairo/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-17Add new version of ccache; update URL (#14942)iarspider1-2/+3
* Add new version of ccache; update URL * Update var/spack/repos/builtin/packages/ccache/package.py Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-02-17Add new version of davix (#14944)iarspider1-2/+2
* Add new version of davix * Update package.py
2020-02-17Add extra version of py-autopep8 (#14970)iarspider1-1/+3
* Add extra version of py-autopep8 * Update package.py
2020-02-17Add extra version of py-dill (#14971)iarspider1-1/+2
* Add extra version of py-dill * Update package.py
2020-02-17Add extra version of py-futures (#14979)iarspider1-1/+3
* Add extra version of py-futures * Update package.py
2020-02-17libcircle depends on a provider of pkgconfig for build (#15029)eugeneswalker1-1/+1
2020-02-17Qt4 patches (#15025)Seth R. Johnson17-57/+42
* Fix patch applicability * Combine patches for missing qt3 headers and remove krell variant The variant should have always been applied. * Restrict QT patches to actual applicable versions/situations - I researched the associated patches so now their `when=` should more closely match when they're actually needed. - I sorted the patch order so they're grouped by version requirement - I renamed the patches so they're listed by version requirements
2020-02-17Add new version of libunwind (#14947)iarspider1-2/+3
* Add new version of libunwind * Update package.py
2020-02-17Added new default tau version: 2.29. Added explicit zlib build requir… ↵wspear1-0/+5
(#15010) * Added new default tau version: 2.29. Added explicit zlib build requirement. Set up environment to use use elf and libz * Changed zlib to link dependency. Removed elf library path load (wasn't able to reproduce the need for this)
generated by cgit v1.2.3-60-g2f50 (git 2.42.0) at 2024-07-04 21:58:10 +0000