summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2019-10-22fetching: S3 upload and download (#11117)Omar Padron21-289/+1420
This extends Spack functionality so that it can fetch sources and binaries from-, push sources and binaries to-, and index the contents of- mirrors hosted on an S3 bucket. High level to-do list: - [x] Extend mirrors configuration to add support for `file://`, and `s3://` URLs. - [x] Ensure all fetching, pushing, and indexing operations work for `file://` URLs. - [x] Implement S3 source fetching - [x] Implement S3 binary mirror indexing - [x] Implement S3 binary package fetching - [x] Implement S3 source pushing - [x] Implement S3 binary package pushing Important details: * refactor URL handling to handle S3 URLs and mirror URLs more gracefully. - updated parse() to accept already-parsed URL objects. an equivalent object is returned with any extra s3-related attributes intact. Objects created with urllib can also be passed, and the additional s3 handling logic will still be applied. * update mirror schema/parsing (mirror can have separate fetch/push URLs) * implement s3_fetch_strategy/several utility changes * provide more feature-complete S3 fetching * update buildcache create command to support S3 * Move the core logic for reading data from S3 out of the s3 fetch strategy and into the s3 URL handler. The s3 fetch strategy now calls into `read_from_url()` Since read_from_url can now handle S3 URLs, the S3 fetch strategy is redundant. It's not clear whether the ideal design is to have S3 fetching functionality in a fetch strategy, directly implemented in read_from_url, or both. * expanded what can be passed to `spack buildcache` via the -d flag: In addition to a directory on the local filesystem, the name of a configured mirror can be passed, or a push URL can be passed directly.
2019-10-22Add dependency version constraint to libgcrypt (#13351)Adam J. Stewart1-1/+1
2019-10-22Fix configure arg to libassuan (#13350)Adam J. Stewart1-3/+4
2019-10-22Fix configure arg to libksba (#13352)Adam J. Stewart1-3/+4
2019-10-22Add latest version of gnupg (#13349)Adam J. Stewart1-13/+23
2019-10-22Add latest version of patchelf (#13348)Adam J. Stewart1-5/+5
2019-10-21Add latest version of zstd (#13340)Adam J. Stewart1-3/+6
* Add latest version of zstd * setup_environment -> setup_build_environment
2019-10-21sniffles: fix zlib header lookup directory. (#13063)Toyohisa Kameyama2-0/+50
* sniffles: fix zlib header lookup directory. * use bamtools and zlib to spack installed package. * remove unneeded patch.
2019-10-21elsi:add CMAKE Fortran MODDIR FLAG for Fujitsu compiler (#13244)noguchi-k1-0/+5
* add CMAKE Fortran MODDIR FLAG for Fujitsu compiler * elsi:remove extra fix
2019-10-21arpack-ng: Fixed error that is not compatible with isnan. (#13246)t-karatsu2-0/+26
2019-10-21bpp-core: Clarify 'isnan' function's namespace. (#13260)t-karatsu2-0/+46
2019-10-21qt: Fix build for newer versions (#13262)Michael Kuhn1-2/+0
For some reason, newer versions of qt fail to build because they cannot find certain system libraries such as libatomic and libdl. Leaving the qmake compiler set to gcc seems to fix the problem. Fixes #13221
2019-10-21New package: Fl (#12877)健美猞猁1-0/+24
2019-10-21Bootstrap environment modules optimizing for generic architectures (#13105)Massimiliano Culpo4-3/+53
fixes #13073 Since #3206 was merged bootstrapping environment-modules was using the architecture of the current host or the best match supported by the default compiler. The former case is an issue since shell integration was looking for a spec targeted at the host microarchitecture. 1. Bootstrap an env modules targeted at generic architectures 2. Look for generic targets in shell integration scripts 3. Add a new entry in Travis to test shell integration
2019-10-21microarchitectures: fix custom compiler versions (#13222)Massimiliano Culpo6-5/+100
Custom string versions for compilers were raising a ValueError on conversion to int. This commit fixes the behavior by trying to detect the underlying compiler version when in presence of a custom string version. * Refactor code that deals with custom versions for better readability * Partition version components with a regex * Fix semantic of custom compiler versions with a suffix * clang@x.y-apple has been special-cased * Add unit tests
2019-10-21Fix Python 3.8 build on macOS (#13338)Adam J. Stewart1-150/+166
2019-10-21Add latest version of py-cffi (#13322)Adam J. Stewart1-5/+10
* Add latest version of py-cffi * setup_environment -> setup_build_environment
2019-10-21Add latest version of py-numpy (#13307)Adam J. Stewart1-4/+5
* Add latest version of py-numpy * setup_environment -> setup_build_environment
2019-10-21Add latest version of py-tables (#13291)Adam J. Stewart1-8/+10
* Add latest version of py-tables * setup_environment -> setup_build_environment
2019-10-21Add latest version of py-torch (#13290)Adam J. Stewart1-28/+30
* Add latest version of py-torch * setup_environment -> setup_build_environment
2019-10-20Use specific version of googletest for uriparser (#13339)Adam J. Stewart1-1/+1
2019-10-20Tut cannot be built with Python 3.7+ (#13341)Adam J. Stewart1-0/+4
2019-10-20Add latest version of proj (#13337)Adam J. Stewart1-1/+2
2019-10-20Add missing zlib dependency to openjpeg (#13336)Adam J. Stewart1-0/+2
2019-10-20openblas: http -> https (#13335)Adam J. Stewart1-2/+2
2019-10-20mapserver: http -> https (#13334)Adam J. Stewart1-1/+1
2019-10-20liblas: http -> https (#13331)Adam J. Stewart1-1/+1
2019-10-20libtiff: http -> https (#13332)Adam J. Stewart1-6/+6
2019-10-20libgeotiff: http -> https (#13330)Adam J. Stewart1-1/+1
2019-10-20Add latest version of libyaml (#13333)Adam J. Stewart1-27/+24
2019-10-20Add latest version of libgcrypt (#13329)Adam J. Stewart1-5/+7
2019-10-20Add latest version of googletest (#13328)Adam J. Stewart1-5/+6
2019-10-20GEOS: http -> https (#13327)Adam J. Stewart1-1/+1
2019-10-20Add missing test dependency to py-alabaster (#13325)Adam J. Stewart1-0/+1
2019-10-20Add latest version of py-typing (#13287)Adam J. Stewart1-3/+3
2019-10-20Add missing dependencies to py-apipkg (#13324)Adam J. Stewart1-0/+3
2019-10-20Add latest version of py-babel (#13323)Adam J. Stewart1-2/+6
2019-10-20Add latest version of py-chardet (#13321)Adam J. Stewart1-1/+4
2019-10-20Add latest version of py-click (#13320)Adam J. Stewart1-2/+5
2019-10-20Add latest version of py-execnet (#13317)Adam J. Stewart1-1/+3
2019-10-20Add latest version of py-decorator (#13318)Adam J. Stewart1-1/+4
2019-10-20Add latest version of py-coverage (#13319)Adam J. Stewart1-1/+3
2019-10-20Add latest version of py-sphinxcontrib-websupport (#13292)Adam J. Stewart1-3/+6
2019-10-20Add latest version of py-sphinx (#13293)Adam J. Stewart1-31/+33
2019-10-20Add latest version of py-snowballstemmer (#13294)Adam J. Stewart1-1/+4
2019-10-20Add latest version of py-requests (#13296)Adam J. Stewart1-6/+10
2019-10-20Add latest version of py-pyyaml (#13297)Adam J. Stewart1-4/+8
2019-10-20Add latest version of py-pytest-xdist (#13298)Adam J. Stewart1-3/+6
2019-10-20Add latest version of py-pytest-runner (#13299)Adam J. Stewart1-1/+2
2019-10-20Add latest version of py-pytest-mock (#13300)Adam J. Stewart1-4/+7