summaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2022-03-17Expand external find for Windows (#27588)John Parent50-423/+504
* Incorporate new search location * Add external user option * proper doc string * Explicit commands in getting started * raise during chgrp on Win recover installer changes Notate admin privleges Windows phase install hooks Find external python and install ninja (#23496) Allow external find python to find windows python and spack install ninja Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17Update tests support for WindowsBetsy McPhail124-193/+1918
Fixup common tests * Remove requirement for Python 2.6 * Skip new failing test Windows: Update url util to handle Windows paths (#27959) * update url util to handle windows paths * Update tests to handle fixed url handling * canonicalize path only when the path type matches the host platform * Skip some url tests on Windows Co-authored-by: Omar Padron <omar.padron@kitware.com> Use threading.TIMEOUT_MAX when available (#24246) This value was introduced in Python 3.2. Specifying a timeout greater than this value will raise an OverflowError. Co-authored-by: Lou Lawrence <lou.lawrence@kitware.com> Co-authored-by: John Parent <john.parent@kitware.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17Add Windows to platform and target changesJohn Parent12-23/+47
Add compiler hint to the root spec for Windows Reporters on Windows (#26038) Reporters use Jinja2 as the templating engine, and Jinja2 indexes templates by Unix separators, even on Windows, so search using Unix paths on all systems. Support patching on win via git (#25871) Handle GRP on windows
2022-03-17Spack on Windows package portsJohn Parent15-58/+93
CMake - Windows Bootstrap (#25825) Remove hardcoded cmake compiler (#26410) Revert breaking cmake changes Ensure no autotools on Windows Perl on Windows (#26612) Python source build windows (#26313) Reconfigure sysconf for Windows Python2.6 compatibility Fxixup new sbang tests for windows Ruby support (#28287) Add NASM support (#28319) Add mock Ninja package for testing
2022-03-17Relocate spack_cmd and scripts from installer to root bin (#24651)John W. Parent6-374/+52
Relocate spack_cmd and scripts from installer to root bin Refactor documentation, installer, and launcher to facilitate that change
2022-03-17Use Python's zipfile, if available (#24556)Betsy McPhail1-3/+21
* Style fixes * Use Python's zipfile, if available The compression libs are optional in Python. Rely on python as a first attempt then fall back to `unzip`
2022-03-17Windows: VS and CMake supportBetsy McPhail3-23/+29
Fix: Building packages with CMake is broken (#24241) vsInstallPaths needs to be renamed vs_install_paths (#24297)
2022-03-17Packaging of netlib-lapack for windows (#24993)Jared Popelar7-103/+190
MSVC's internal CMake and Ninja now detected by spack external find and added to packages.yaml Saving progress on packaging zlib for Windows Fixing the shared CMake flag * Loading Intel's ifx Fortran compiler into MSVC; if there are multiple versions of MSVC installed and detected, ifx will only be placed into the first block written in compilers.yaml. The version number of ifx can be detected using MSVC's version flag (instead of /QV) by using ignore_version_errors. This commit also provides support for detection of Intel compilers in their own compiler block by adding ifx.exe to the fc/f77_name blocks inside intel.py * Giving CMake a Fortran compiler argument * Adding patch file for removing duplicated mangling header for versions 3.9.1 and older; static and shared now successfully building on Windows * Have netlib-lapack depend on ninja@1.10 Co-authored-by: John R. Cary <cary@txcorp.com> Co-authored-by: Jared Popelar <jpopelar@txcorp.com> Making a default config.yaml for Windows Small path length for build_stage Provide more prerequisite details, mention default config.yaml Killing an unnecessary setvars call Replacing some lost changes, proofreading, updating windows-supported package list Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-17Windows: Create installer and environmentlou.lawrence@kitware.com18-11/+987
* Add 'make-installer' command for Windows * Add '--bat' arg to env activate, env deactivate and unload commands * An equivalent script to setup-env on linux: spack_cmd.bat. This script has a wrapper to evaluate cd, load/unload, env activate/deactivate.(#21734) * Add spacktivate and config editor (#22049) * spack_cmd: will find python and spack on its own. It preferentially tries to use python on your PATH (#22414) * Ignore Windows python installer if found (#23134) * Bundle git in windows installer (#23597) * Add Windows section to Getting Started document (#23131), (#23295), (#24240) Co-authored-by: Stephen Crowell <stephen.crowell@kitware.com> Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com> Co-authored-by: Jared Popelar <jpopelar@txcorp.com> Co-authored-by: Ben Cowan <benc@txcorp.com> Update Installer CI Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-17MSVC Compiler Find and vcvars env integration (#22856)Betsy McPhail3-2/+86
Made the vcvars batch script location a member variable of the msvc compiler subclass, initialized from the compiler executable path. Added a setup_custom_environment() method to the msvc subclass that sources the vcvars script, dumps the environment, and copies the relevant environment variables to the Spack environment. Added class variables to the Windows OS and MSVC compiler subclasses to enable finding the compiler executables and determining their versions.
2022-03-17use pytest stdout/err capture (#22584)loulawrence3-44/+59
* On windows, write to StringIO without redirection in test cases to avoid conflicting with logger
2022-03-17Features/windows install (#22204)Ben Cowan4-25/+50
* Fixed path and uid issues. * Added needed import statement; kluged .exe extension. * Got package to build. Some manual intervention necessary, including sourcing the MSVC setup script and having certain configuration parameters. * Removed CMake executable suffix hack.
2022-03-17Allow 'spack external find' to find executables on the system path (#22091)Betsy McPhail5-51/+247
Co-authored-by: Lou Lawrence <lou.lawrence@kitware.com>
2022-03-17Windows: Symlink supportBetsy McPhail20-44/+224
To provide Windows-compatible functionality, spack code should use llnl.util.symlink instead of os.symlink. On non-Windows platforms and on Windows where supported, os.symlink will still be used. Use junctions when symlinks aren't supported on Windows (#22583) Support islink for junctions (#24182) Windows: Update llnl/util/filesystem * Use '/' as path separator on Windows. * Recognizing that Windows paths start with '<Letter>:/' instead of '/' Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-17Create rename utility functionBetsy McPhail11-23/+45
os.rename() fails on Windows if file already exists. Create getuid utility function (#21736) On Windows, replace os.getuid with ctypes.windll.shell32.IsUserAnAdmin(). Tests: Use getuid util function Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17Workarounds for install errors on Windows (#21890)Betsy McPhail3-22/+32
1. Forwarding sys.stdin, e.g. use input_multiprocess_fd, gives an error on Windows. Skipping for now 3. subprocess_context needs to serialize for Windows, like it does for Mac. Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com> Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-17Adding basic Windows features (#21259)Ben Cowan6-3/+138
* Snapshot of some MSVC infrastructure added during experiments a while ago. Rebasing from spack/develop. * Added platform and OS definitions for Windows. * Updated Windows platform file to conform to new archspec use. * Added Windows as a platform; introduced some debugging code. * Added type annotations. * Fixed copyright. * Removed print statements. * Ensure `spack arch` returns correctly on Windows (#21428) * Correctly identify windows as 'windows-Windows10-AMD64'
2022-03-17PackageViewMixin: fix symlinks conflict issue (#29515)Harmen Stoppels1-2/+2
`stat`'ing a file in the dst dir is the wrong thing to do, you should `lstat` to capture broken symlinks.
2022-03-15optimize instantiation and comparison of versions (#29429)Tom Scogland1-4/+14
Re-work the checks and comparisons around commit versions, when no commit version is involved the overhead is now in the noise, where one is the overhead is now constant rather than linear.
2022-03-14mpich: fix the detection of the package (#29284)Sergey Kosukhin1-2/+2
2022-03-14compiler.py: early return in compiler_environment when no modules (#29441)Harmen Stoppels1-0/+5
2022-03-13Remove "setup_environment" and "setup_dependent_environment" (#29463)Massimiliano Culpo1-31/+4
fixes #29446 The new setup_*_environment functions have been falling back to calling the old functions and warn the user since #11115. This commit removes the fallback behavior and any use of: - setup_environment - setup_dependent_environment in the codebase
2022-03-11module_cmd.py: use posix awk; fix stderr redirection (#29440)Harmen Stoppels2-62/+34
Emulates `env -0` in a posix compliant way, avoiding a slow python process, speeds up setting up the build env when modules should load.
2022-03-10intel-mkl: load compiler modules when querying compiler (#29439)Harmen Stoppels2-14/+16
2022-03-10Allow for multiple dependencies/dependents from the same package (#28673)Massimiliano Culpo13-320/+774
Change the internal representation of `Spec` to allow for multiple dependencies or dependents stemming from the same package. This change permits to represent cases which are frequent in cross compiled environments or to bootstrap compilers. Modifications: - [x] Substitute `DependencyMap` with `_EdgeMap`. The main differences are that the latter does not support direct item assignment and can be modified only through its API. It also provides a `select_by` method to query items. - [x] Reworked a few public APIs of `Spec` to get list of dependencies or related edges. - [x] Added unit tests to prevent regression on #11983 and prove the synthetic construction of specs with multiple deps from the same package. Since #22845 went in first, this PR reuses that format and thus it should not change hashes. The same package may be present multiple times in the list of dependencies with different associated specs (each with its own hash).
2022-03-09environment.py: allow link:run (#29336)Harmen Stoppels6-98/+159
* environment.py: allow link:run Some users want minimal views, excluding run-type dependencies, since those type of dependencies are covered by rpaths and the symlinked libraries in the view aren't used anyways. With this change, an environment like this: ``` spack: specs: ['py-flake8'] view: default: root: view link: run ``` includes python packages and python, but no link type deps of python.
2022-03-09Hide package repository count when redirecting stdout (#29402)百地 希留耶1-3/+5
2022-03-09Fix tab completion erroring with `spack unit-test` (#29405)百地 希留耶1-1/+1
2022-03-09ci.py: fix utf-8 issue with git log output (#29419)Harmen Stoppels1-2/+4
2022-03-08version.py: set is_commit in constructor (#29369)Harmen Stoppels1-10/+3
Speeds up comparison on `Version` by ~2.5x, e.g. ```python In [1]: v = spack.version.Version('1.0.0'); w = spack.version.Version('1.0.2') In [2]: %timeit v < w 1.47 µs ± 5.59 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) 535 ns ± 1.75 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) ```
2022-03-05find: deprecate `spack find -bootstrap` for `spack -bootstrap find` (#29152)Todd Gamblin2-0/+11
Since adding the `spack --bootstrap` argument we don't need `spack find --bootstrap` anymore. Deprecate it.
2022-03-04Add a testHarmen Stoppels1-0/+8
2022-03-04VersionRange.satisfies should test non-empty intersectionHarmen Stoppels1-25/+13
2022-03-04Fix importing Spack packages as Python modules (#29221)Massimiliano Culpo9-53/+82
fixes #29203 This PR fixes a subtle bug we have when importing Spack packages as Python modules that can lead to multiple module objects being created for the same package. It also fixes all the places in unit-tests where "relying" on the old bug was crucial to have a new "clean" state of the package class.
2022-03-03Revert GCS fetch strategy, to remove s3 interface (#28736)Doug Jacobsen1-7/+1
This commit reverts the GCS fetch strategy to before commit: d7596125231e800ca41c60e379be2b8abb47d20d The previous commit added some s3 syntax to handle connections, but added them into the GCS fetch strategy in a way that prevents GCS from working anymore.
2022-03-03rocmcc compiler: initial commit based on aocc and clang (#28575)Greg Becker7-5/+91
* rocmcc compiler: initial commit based on aocc and clang Co-authored-by: luker <luke.roskop@hpe.com> Co-authored-by: Tom Scogland <scogland1@llnl.gov>
2022-03-03installer: Fix cosmetic problem with terminal title (#29070)Michael Kuhn1-4/+8
The status displayed in the terminal title could be wrong when doing distributed builds. For instance, doing `spack install glib` in two different terminals could lead to the current package being reported as `40/29` due to the way Spack handles retrying locks. Work around this by keeping track of the package IDs that were already encountered to avoid counting packages twice.
2022-03-03test/installer.py: remove commented code and inaccurate docstring (#29305)Scott Wittenburg1-21/+1
2022-03-02make @llnl.util.lang.memoized support kwargs (#21722)Danny McClanahan2-10/+96
* make memoized() support kwargs * add testing for @memoized
2022-03-02Fix typos when forwarding arguments to traverse_edges (#29261)Massimiliano Culpo4-4/+11
A few calls use `deptypes=...` instead of `deptype=...`
2022-03-01test_env_install_two_specs_same_dep: properly check installed specs (#29222)Tamara Dahlgren1-4/+4
2022-02-25commands: Propgate command return value as exit code (#29223)Scott Wittenburg1-1/+1
2022-02-23Testing: Summarize test results and add verbose output (#28700)Tamara Dahlgren6-13/+106
2022-02-23Move early exit for setup only argument (#29041)Massimiliano Culpo1-4/+4
See https://github.com/spack/spack/pull/28468/files#r809156986 If we exit before generating the: error("Dependencies must have compatible OS's with their dependents"). ... facts we'll output a problem that is effectively different by the one solved by clingo.
2022-02-23Checksum match (#28989)Tom Scogland2-2/+25
* cmd/checksum: prefer url matching url_from_version This is a minimal change toward getting the right archive from places like github. The heuristic is: * if an archive url exists, take its version * generate a url from the package with pkg.url_from_version * if they match * stop considering other URLs for this version * otherwise, continue replacing the url for the version I doubt this will always work, but it should address a variety of versions of this bug. A good test right now is `spack checksum gh`, which checksums macos binaries without this, and the correct source packages with it. fixes #15985 related to #14129 related to #13940 * add heuristics to help create as well Since create can't rely on an existing package, this commit adds another pair of heuristics: 1. if the current version is a specifically listed archive, don't replace it 2. if the current url matches the result of applying `spack.url.substitute_version(a, ver)` for any a in archive_urls, prefer it and don't replace it fixes #13940 * clean up style and a lingering debug import * ok flake8, you got me * document reference_package argument * Update lib/spack/spack/util/web.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * try to appease sphinx Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2022-02-22Add `spack --bootstrap` option for accessing bootstrap store (#25601)Todd Gamblin5-58/+64
We can see what is in the bootstrap store with `spack find -b`, and you can clean it with `spack clean -b`, but we can't do much else with it, and if there are bootstrap issues they can be hard to debug. We already have `spack --mock`, which allows you to swap in the mock packages from the command line. This PR introduces `spack -b` / `spack --bootstrap`, which runs all of spack with `ensure_bootstrap_configuration()` set. This means that you can run `spack -b find`, `spack -b install`, `spack -b spec`, etc. to see what *would* happen with bootstrap configuration, to remove specific bootstrap packages, etc. This will hopefully make developers' lives easier as they deal with bootstrap packages. This PR also uses a `nullcontext` context manager. `nullcontext` has been implemented in several other places in Spack, and this PR consolidates them to `llnl.util.lang`, with a note that we can delete the function if we ever reqyire a new enough Python. - [x] introduce `spack --bootstrap` option - [x] consolidated all `nullcontext` usages to `llnl.util.lang`
2022-02-21bugfix: Not all concrete versions on the CLI should be considered real (#28620)Todd Gamblin5-24/+69
Some "concrete" versions on the command line, e.g. `qt@5` are really meant to satisfy some actual concrete version from a package. We should only assume the user is introducing a new, unknown version on the CLI if we, well, don't know of any version that satisfies the user's request. So, if we know about `5.11.1` and `5.11.3` and they ask for `5.11.2`, we'd ask the solver to consider `5.11.2` as a solution. If they just ask for `5`, though, `5.11.1` or `5.11.3` are fine solutions, as they satisfy `@5`, so use them. Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-02-21Bring back cuda@11.4.0 conflicts for GCC and clang; add 11.4.3:11.4.4 (#29076)Alberto Invernizzi1-0/+2
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-02-18spack external find: change default behavior (#29031)Massimiliano Culpo2-4/+22
See https://github.com/spack/spack/issues/25353#issuecomment-1041868116 This commit changes the default behavior of ``` $ spack external find ``` from searching all the possible packages Spack knows about to search only for the ones tagged as being a "build-tool". It also introduces a `--all` option to restore the old behavior.
2022-02-17darwin: robust macos version detection (#28991)Seth R. Johnson2-7/+49
Prefer `sw_vers` to `platform.mac_ver`. In anaconda3 installation, for example, the latter reports 10.16 on Monterey -- I think this is affected by how and where the python instance was built. Use MACOSX_DEPLOYMENT_TARGET if present to override the operating system choice.