summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2022-03-18Add command for reading a json-based DB description (#24894)Peter Scheibel14-41/+712
Adds `spack external read-cray-manifest`, which reads a json file that describes a set of package DAGs. The parsed results are stored directly in the database. A user can see these installed specs with `spack find` (like any installed spec). The easiest way to use them right now as dependencies is to run `spack spec ... ^/hash-of-external-package`. Changes include: * `spack external read-cray-manifest --file <path/to/file>` will add all specs described in the file to Spack's installation DB and will also install described compilers to the compilers configuration (the expected format of the file is described in this PR as well including examples of the file) * Database records now may include an "origin" (the command added in this PR registers the origin as "external-db"). In the future, it is assumed users may want to be able to treat installs registered with this command differently (e.g. they may want to uninstall all specs added with this command) * Hash properties are now always preserved when copying specs if the source spec is concrete * I don't think the hashes of installed-and-concrete specs should change and this was the easiest way to handle that * also specs that are concrete preserve their `.normal` property when copied (external specs may mention compilers that are not registered, and without this change they would fail in `normalize` when calling `validate_or_raise`) * it might be this should only be the case if the spec was installed - [x] Improve testing - [x] Specifically mark DB records added with this command (so that users can do something like "uninstall all packages added with `spack read-external-db`) * This is now possible with `spack uninstall --all --origin=external-db` (this will remove all specs added from manifest files) - [x] Strip variants that are listed in json entries but don't actually exist for the package Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-03-18hdf5-vol-log: new version 1.1 (#29553)Larry Knox1-2/+4
2022-03-18PETSc: split install() into configure(), build(), install() (#29126)JDBetteridge1-5/+18
2022-03-18hepmcanalysis: propagate cxxstd from root (#29512)haralmha1-8/+5
* Use same cxx value as root * Remove pointer syntax from non-pointer type in source * Run patch function before build * Use raw string in filter_file and merge edit function with patch * Escape parentheses * Use gDirectory from ROOT instead of CurrentDirectory function
2022-03-18mmg: add 5.6.0 (#29534)Carlos Bederián1-0/+1
2022-03-18environment: Don't print 'updating view at' when no specs (#29536)Harmen Stoppels1-1/+2
2022-03-18Split the workflow section and remove outdated advices (#29344)Massimiliano Culpo7-1248/+287
This PR removes a few outdated sections from the "Basics" part of the documentation. It also makes a few topic under the environment section more prominent by removing an unneeded spack.yaml subsection and promoting everything under it.
2022-03-17Make boost minimal and composable (Original PR#22303) (#28623)Abhik Sarkar190-104/+1117
* Make boost composable Currently Boost enables a few components through variants by default, which means that if you want to use only what you need and no more, you have to explicitly disable these variants, leading to concretization errors whenever a second package explicitly needs those components. For instance if package A only needs `+component_a` it might depend on `boost +component_a ~component_b`. And if packge B only needs `+component_b` it might depend on `boost ~component_a +component_b`. If package C now depends on both A and B, this leads to unsatisfiable variants and hence a concretization error. However, if we default to disabling all components, package A can simply depend on `boost +component_a` and package B on `boost +component_b` and package C will concretize to depending on `boost +component_a +component_b`, and whatever you install, you get the bare minimum. * Fix style * Added composable boost dependencies for folly * fixing akantu merge issue * hpctoolkit boost dependencies already defined * Fix Styles * Fixup style once more * Adding isort fix * isort one more time * Fix for package audit issue Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com> Co-authored-by: Ryan O'Malley <rd.omalley@comcast.net>
2022-03-17clinfo: add new versions (#29537)lorddavidiii1-0/+2
2022-03-17Add deprecation warnings for activate/deactivate/view (#29430)Harmen Stoppels2-0/+8
2022-03-17windows ci: make conditional on style tests etc (#29567)Harmen Stoppels1-0/+4
2022-03-17Remove references to `features/windows-support` branch (#29565)Zack Galbreath2-6/+1
2022-03-17New package: py-jaconv (#29548)Jen Herting1-0/+19
* [py-jaconv] created template * [py-jaconv] - added homepage - added description - depends on setuptools - removed fixmes
2022-03-17octa: new package. (#29333)Toyohisa Kameyama4-0/+185
2022-03-17PWSH SupportJohn Parent1-0/+59
Include support for powershell Prepend cmd and pwsh with [spack] to denote spack enabled shell
2022-03-17Windows Support: Testing Suite integrationJohn Parent102-1065/+770
Broaden support for execution of the test suite on Windows. General bug and review fixups
2022-03-17Failed _write now track that the DB is inconsistentPeter Josef Scheibel1-0/+15
Consistency is restored on next transaction
2022-03-17"spack commands --update-completion"John Parent105-1061/+687
2022-03-17Path handling (#28402)John W. Parent36-232/+493
Consolidate Spack's internal filepath logic to a select few places and refactor to consistent internal useage of os.path utilities. Creates a prefix, and a series of utilities in the path utility module that facilitate handling paths in a platform agnostic manner. Convert Windows paths to posix paths internally Prefer posixpath.join instead of os.path.join Updated util/ directory to account for Windows integration Co-authored-by: Stephen Crowell <stephen.crowell@khq.kitware.com> Co-authored-by: John Parent <john.parent@kitware.com> Module template format for windows (#23041)
2022-03-17Expand external find for Windows (#27588)John Parent56-457/+535
* Incorporate new search location * Add external user option * proper doc string * Explicit commands in getting started * raise during chgrp on Win recover installer changes Notate admin privleges Windows phase install hooks Find external python and install ninja (#23496) Allow external find python to find windows python and spack install ninja Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17Update tests support for WindowsBetsy McPhail126-200/+1925
Fixup common tests * Remove requirement for Python 2.6 * Skip new failing test Windows: Update url util to handle Windows paths (#27959) * update url util to handle windows paths * Update tests to handle fixed url handling * canonicalize path only when the path type matches the host platform * Skip some url tests on Windows Co-authored-by: Omar Padron <omar.padron@kitware.com> Use threading.TIMEOUT_MAX when available (#24246) This value was introduced in Python 3.2. Specifying a timeout greater than this value will raise an OverflowError. Co-authored-by: Lou Lawrence <lou.lawrence@kitware.com> Co-authored-by: John Parent <john.parent@kitware.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17Add Windows to platform and target changesJohn Parent12-23/+47
Add compiler hint to the root spec for Windows Reporters on Windows (#26038) Reporters use Jinja2 as the templating engine, and Jinja2 indexes templates by Unix separators, even on Windows, so search using Unix paths on all systems. Support patching on win via git (#25871) Handle GRP on windows
2022-03-17Spack on Windows package portsJohn Parent27-199/+648
CMake - Windows Bootstrap (#25825) Remove hardcoded cmake compiler (#26410) Revert breaking cmake changes Ensure no autotools on Windows Perl on Windows (#26612) Python source build windows (#26313) Reconfigure sysconf for Windows Python2.6 compatibility Fxixup new sbang tests for windows Ruby support (#28287) Add NASM support (#28319) Add mock Ninja package for testing
2022-03-17Add Github Actions for Windows (#24504)John Parent11-8/+262
Setup Installer CI (#25184), (#25191) Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com> Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17Relocate spack_cmd and scripts from installer to root bin (#24651)John W. Parent6-81/+63
Relocate spack_cmd and scripts from installer to root bin Refactor documentation, installer, and launcher to facilitate that change
2022-03-17Use Python's zipfile, if available (#24556)Betsy McPhail1-3/+21
* Style fixes * Use Python's zipfile, if available The compression libs are optional in Python. Rely on python as a first attempt then fall back to `unzip`
2022-03-17Windows: VS and CMake supportBetsy McPhail4-30/+33
Fix: Building packages with CMake is broken (#24241) vsInstallPaths needs to be renamed vs_install_paths (#24297)
2022-03-17Packaging of netlib-lapack for windows (#24993)Jared Popelar11-123/+297
MSVC's internal CMake and Ninja now detected by spack external find and added to packages.yaml Saving progress on packaging zlib for Windows Fixing the shared CMake flag * Loading Intel's ifx Fortran compiler into MSVC; if there are multiple versions of MSVC installed and detected, ifx will only be placed into the first block written in compilers.yaml. The version number of ifx can be detected using MSVC's version flag (instead of /QV) by using ignore_version_errors. This commit also provides support for detection of Intel compilers in their own compiler block by adding ifx.exe to the fc/f77_name blocks inside intel.py * Giving CMake a Fortran compiler argument * Adding patch file for removing duplicated mangling header for versions 3.9.1 and older; static and shared now successfully building on Windows * Have netlib-lapack depend on ninja@1.10 Co-authored-by: John R. Cary <cary@txcorp.com> Co-authored-by: Jared Popelar <jpopelar@txcorp.com> Making a default config.yaml for Windows Small path length for build_stage Provide more prerequisite details, mention default config.yaml Killing an unnecessary setvars call Replacing some lost changes, proofreading, updating windows-supported package list Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-17Windows: Create installer and environmentlou.lawrence@kitware.com20-12/+1002
* Add 'make-installer' command for Windows * Add '--bat' arg to env activate, env deactivate and unload commands * An equivalent script to setup-env on linux: spack_cmd.bat. This script has a wrapper to evaluate cd, load/unload, env activate/deactivate.(#21734) * Add spacktivate and config editor (#22049) * spack_cmd: will find python and spack on its own. It preferentially tries to use python on your PATH (#22414) * Ignore Windows python installer if found (#23134) * Bundle git in windows installer (#23597) * Add Windows section to Getting Started document (#23131), (#23295), (#24240) Co-authored-by: Stephen Crowell <stephen.crowell@kitware.com> Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com> Co-authored-by: Jared Popelar <jpopelar@txcorp.com> Co-authored-by: Ben Cowan <benc@txcorp.com> Update Installer CI Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-17MSVC Compiler Find and vcvars env integration (#22856)Betsy McPhail3-2/+86
Made the vcvars batch script location a member variable of the msvc compiler subclass, initialized from the compiler executable path. Added a setup_custom_environment() method to the msvc subclass that sources the vcvars script, dumps the environment, and copies the relevant environment variables to the Spack environment. Added class variables to the Windows OS and MSVC compiler subclasses to enable finding the compiler executables and determining their versions.
2022-03-17use pytest stdout/err capture (#22584)loulawrence3-44/+59
* On windows, write to StringIO without redirection in test cases to avoid conflicting with logger
2022-03-17Features/windows install (#22204)Ben Cowan4-25/+50
* Fixed path and uid issues. * Added needed import statement; kluged .exe extension. * Got package to build. Some manual intervention necessary, including sourcing the MSVC setup script and having certain configuration parameters. * Removed CMake executable suffix hack.
2022-03-17Allow 'spack external find' to find executables on the system path (#22091)Betsy McPhail5-51/+247
Co-authored-by: Lou Lawrence <lou.lawrence@kitware.com>
2022-03-17Windows: Symlink supportBetsy McPhail20-44/+224
To provide Windows-compatible functionality, spack code should use llnl.util.symlink instead of os.symlink. On non-Windows platforms and on Windows where supported, os.symlink will still be used. Use junctions when symlinks aren't supported on Windows (#22583) Support islink for junctions (#24182) Windows: Update llnl/util/filesystem * Use '/' as path separator on Windows. * Recognizing that Windows paths start with '<Letter>:/' instead of '/' Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-17Create rename utility functionBetsy McPhail11-23/+45
os.rename() fails on Windows if file already exists. Create getuid utility function (#21736) On Windows, replace os.getuid with ctypes.windll.shell32.IsUserAnAdmin(). Tests: Use getuid util function Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17Workarounds for install errors on Windows (#21890)Betsy McPhail4-24/+43
1. Forwarding sys.stdin, e.g. use input_multiprocess_fd, gives an error on Windows. Skipping for now 3. subprocess_context needs to serialize for Windows, like it does for Mac. Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-17Adding basic Windows features (#21259)Ben Cowan6-3/+138
* Snapshot of some MSVC infrastructure added during experiments a while ago. Rebasing from spack/develop. * Added platform and OS definitions for Windows. * Updated Windows platform file to conform to new archspec use. * Added Windows as a platform; introduced some debugging code. * Added type annotations. * Fixed copyright. * Removed print statements. * Ensure `spack arch` returns correctly on Windows (#21428) * Correctly identify windows as 'windows-Windows10-AMD64'
2022-03-17assimp: depends on pkgconfig; disable bundled dependencies (#29551)Wouter Deconinck1-0/+4
* [assimp] depends_on pkgconfig and disable hunter, build zlib/minizip
2022-03-17openssh: Update to version 8.9 (#29539)Erik Schnetter1-0/+1
2022-03-17PackageViewMixin: fix symlinks conflict issue (#29515)Harmen Stoppels1-2/+2
`stat`'ing a file in the dst dir is the wrong thing to do, you should `lstat` to capture broken symlinks.
2022-03-17fastjet: use mirror for sources (#29357)Valentin Volkl1-1/+5
as a workaround for https://github.com/spack/spack/issues/20050
2022-03-17Add intermediate folder in lib to LD_LIBRARY_PATH (#29519)haralmha1-0/+3
2022-03-17openssl: Deprecate old versions (#29554)Erik Schnetter1-2/+2
2022-03-17New package: py-quart (#29142)Jen Herting1-0/+29
* [py-quart] created template * [py-quart] - added homepage - added description - added dependencies - removed fixmes * [py-quart] removed setuptools dependency
2022-03-17assimp: remove -Werror to allow compilation on Intel oneAPI (#29552)Wouter Deconinck1-0/+3
2022-03-17python: allow versions with garbage suffix (#29543)Tom Scogland1-1/+3
* python: allow versions with garbage suffix Ubuntu 22.04 preview python prints version as 3.10.2+, the + causes version parsing to fail and breaks detection. * Add version comment * match VALID_VERSION regex
2022-03-16libiconv,lmod: improvements for static builds (#29493)Seth R. Johnson2-1/+2
* libiconv: compile with pic even when static build * lmod: require shared lua It seems to be unable to detect lua-posix when using a static lua: ``` Error: The follow lua module(s) are missing: posix ```
2022-03-16openssl: Security update (#29538)Erik Schnetter1-1/+3
2022-03-16[py-configparser] added version 5.2.0 (#29136)Jen Herting1-2/+7
* [py-configparser] added version 5.2.0 * [py-configparser] fixed setuptools versions and added setuptools_scm
2022-03-16New package: py-hypercorn (#29141)Jen Herting1-0/+25
* [py-hypercorn] created template * [py-hypercorn] - added homepage - added description - added dependencies - removed fixmes * [py-hypercorn] removed setuptools dependency and added version restriction for py-typing-extensions