Age | Commit message (Collapse) | Author | Files | Lines |
|
The config.guess detection used a relative path that did not work in
combination with `check_call`. Use an absolute path instead.
|
|
* cray: detect frontend compilers automatically
This commit permits to detect frontend compilers
automatically, with the exception of cce.
Co-authored-by: Gregory Becker <becker33.llnl.gov>
|
|
[george.hartzell@172-16-193-97 spack-explore-docker]$ spack containerize
Running `spack containerize` with the example `spack.yaml` file fails
with an error that ends like so:
```
[...]
File "/local_scratch/hartzell/tmp/spack-explore-docker/lib/spack/external/ruamel/yaml/scanner.py", line 165, in need_more_tokens
self.stale_possible_simple_keys()
File "/local_scratch/hartzell/tmp/spack-explore-docker/lib/spack/external/ruamel/yaml/scanner.py", line 309, in stale_possible_simple_keys
"could not find expected ':'", self.get_mark())
ruamel.yaml.scanner.ScannerError: while scanning a simple key
in "/local_scratch/hartzell/tmp/spack-explore-docker/spack.yaml", line 26, column 1
could not find expected ':'
in "/local_scratch/hartzell/tmp/spack-explore-docker/spack.yaml", line 28, column 5
```
Indenting the block string fixes the problem for me.
CentOS 7,
```
$ spack --version
0.14.2-1529-ec58f28c2
```
|
|
|
|
* Add support for macOS Big Sur
* Big Sur is actually macOS 11.0
|
|
* env: no automatic activation
* Ensure ci rebuild jobs activate the environment (no longer automagic)
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
|
|
* Start moving toward a json buildcache index
* Add spec and database index schemas
* Add a schema for buildcache spec.yaml files
* Provide a mode for database class to generate buildcache index
* Update db and ci tests to validate object w/ new schema
* Remove unused temporary upload-s3 command
* Use database class to generate buildcache index
* Do not generate index with each buildcache creation
* Make buildcache index mode into a couple of constructor args to Database class
* Use keyword args for _createtarball
* Parse new json index when we get specs from buildcache
Now that only one index file per mirror needs to be fetched in
order to have all the concrete specs for binaries available on the
mirror, we can just fetch and refresh the cached specs every time
instead of needing to use the '-f' flag to force re-reading.
|
|
* build_environment: verify compiler executables exist and are accessible
* fix existing tests
* test compiler executable verification
|
|
|
|
* First fix for SPACK_DEPENDENCIES problem when doing setup
* Get rid of transitive include path in setup.
* Export SPACK_INCLUDE_DIRS into spconfig.py
|
|
(#17275)
|
|
* add buildcache create test
* add functionality and test to create buildcache from environment
* use env.concretized_user_specs rather than env.roots to get concretized specs, as suggested in review from becker33
|
|
* Allow `spack remove -f` and `spack uninstall` to work on matrices
Allow Environment.remove(force=True) to remove the concrete spec from the environment
even when the user spec cannot be removed because it is in a matrix.
|
|
* cce: detect modules based on the classic backend
* cce: tweaked version checks for clang based options
* Added unit test for cce flags
|
|
* add workaround for gitlab ci needs limit
* fix style/address review comments
* convert filter obj to list
* update command completion
* remove dict comprehension
* add workaround tests
* fix sorting issue between disparate types
* add indeces to format
|
|
* prevent multiple version sigils in the same spec
* fix packages with malformed versions
|
|
clauses (#17056)
|
|
* Separate Apple Clang from LLVM Clang
Apple Clang is a compiler of its own. All places
referring to "-apple" suffix have been updated.
* Hack to use a dash in 'apple-clang'
To be able to use autodoc from Sphinx we need
a valid Python name for the module that contains
Apple's Clang code.
* Updated packages to account for the existence of apple-clang
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Added unit test for XCode related functions
Co-authored-by: Gregory Becker <becker33@llnl.gov>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
|
|
* short-circuit is_activated check when the extendee is installed upstream
* add test for checking activation status of packages with an extendee installed upstream
|
|
spack config add <value>: add nested value value to the configuration scope specified
spack config remove/rm: remove specified configuration from the relevant scope
|
|
* Add ability to force removal of install failure tracking data through spack clean
* Add clean failures option to packaging guide
|
|
packages. (#8623)
|
|
* Some minor fixes to set_permissions() in file_permissions.py
The set_permissions() routine claims to prevent users from creating
world writable suid binaries. However, it seems to only be checking
for/preventing group writable suid binaries.
This patch modifies the routine to check for both world and group
writable suid binaries, and complain appropriately.
* permissions.py: Add test to check blocks world writable SUID files
The original test_chmod_rejects_group_writable_suid tested
that the set_permissions() function in
lib/spack/spack/util/file_permissions.py
would raise an exception if changed permission on a file with
both SUID and SGID plus sticky bits is chmod-ed to g+rwx and o+rwx.
I have modified so that more narrowly tests a file with SUID
(and no SGID or sticky bit) set is chmod-ed to g+w.
I have added a second test test_chmod_rejects_world_writable_suid
that checks that exception is raised if an SUID file is chmod-ed
to o+w
* file_permissions.py: Raise exception when try to make sgid file world writable
Updated set_permissions() in file_permissions.py to also raise
an exception if try to make an SGID file world writable. And
added corresponding unit test as well.
* Remove debugging prints from permissions.py
|
|
|
|
* Module index should not be unconditionally overwritten
Uncovered after we switched our CI to generate modules for packages
one-by-one rather than in bulk. This overwrote a complete module index
with an index with a single entry, and broke our downstream Spack
instances that needed the upstream module index.
|
|
|
|
|
|
bugfix: fix compiler environment handling to reset environment after
|
|
* add initial optimization script
* integrate optimization in spack ci
* make optimization opt-in
* fix import error
* flake8 fixes
* update command completion
* work around vermin errors
* fix sphynx errors
|
|
* swig: revise 'autotools' automated 'config.*' update mechanism to support 'config.sub' and adapt 'swig' accordingly
|
|
* Fetching from urls: Error Message
Fix the error message when fetching from consecutive `urls` of a
package version. Each fail should show the currently failing URL,
not the first url.
Example multi-problem run that occured in real life:
```
==> 5821: Installing util-macros
curl: (28) Connection timed out after 10000 milliseconds
curl: (16) Error in the HTTP2 framing layer
curl: (22) The requested URL returned error: 403 Forbidden
==> Fetching https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2
==> Failed to fetch file from URL: https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2
Curl failed with error 28
==> Fetching https://mirrors.ircam.fr/pub/x.org/individual/util/util-macros-1.19.1.tar.bz2
==> Failed to fetch file from URL: https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2
Curl failed with error 16
==> Fetching http://xorg.mirrors.pair.com/individual/util/util-macros-1.19.1.tar.bz2
==> Failed to fetch file from URL: https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2
URL https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2 was not found!
==> Fetching from https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2 failed.
==> Error: FetchError: All fetchers failed for spack-stage-util-macros-1.19.1-se2a2e74oyusj2r4esgcb7pr3qhh45ef
```
- `urls[0]`: HTTP2 layer error
- `urls[1]`: timeout
- `urls[2]`: missing file on mirror
* x.org: two more mirrors
x.org mirrors are a bit tricky, since many are out-of-sync or off.
A good package to test with is `util-macros`, which had a "recent"
release.
|
|
|
|
Co-authored-by: Gregory Becker <becker33.llnl.gov>
|
|
Link to the E4S build cache in the according section of the docs.
|
|
* Catch yaml filename corner case in spec parser
* improved regex and remove redundant error checking
Co-authored-by: Gregory Becker <becker33@llnl.gov>
|
|
* Skip failing test on MacOS
* Update setup-python action and unpin coverage
|
|
|
|
* Fix bug in binary string replacement that results in padding being added multiple times
* Update comment
* Update comment again
|
|
|
|
Tests on version detection are now separate from other
tests done on compiler.
|
|
Without this the test would leave spurious "spack-build-*.txt"
files in the current working directory.
|
|
* Changed the 'include' config section to use 'substitute_path_variables' to allow for Spack config variables to be used (e.g. $spack).
* Fixed a bug with 'include' section path expansion and added a test case for 'include' paths with embedded config variables.
|
|
|
|
* SpecList: fix and refactor variable expansion
|
|
* Cray: fix Blue Waters support
* pkg-config env vars needed on Blue Waters
* cray platform: fix support for user-build MPI on cray machines
* reintroduce cray environment cleaning behind cnl version guard
* cray platform: fix support for user-build MPI on cray machines
Co-authored-by: Gregory <becker33@llnl.gov>
|
|
view (#16955)
Bugfix for hardlinks and copies
|
|
Builds can be stopped before the final install phase due to user requests. Those builds
should not be registered as installed in the database.
We had code intended to handle this but:
1. It caught the wrong type of exception
2. We were catching these exceptions to suppress them at a lower level in the stack
This PR allows the StopIteration to propagate through a ChildError, and catches it
properly. Also added to an existing test to prevent regression.
|
|
This fixes a fork bomb in `spack versions`. Recursive generation of pools
to scrape URLs in `_spider` was creating large numbers of processes.
Instead of recursively creating process pools, we now use a single
`ThreadPool` with a concurrency limit.
More on the issue: having ~10 users running at the same time spack
versions on front-end nodes caused kernel lockup due to the high number
of sockets opened (sys-admin reports ~210k distributed over 3 nodes).
Users were internal, so they had ulimit -n set to ~70k.
The forking behavior could be observed by just running:
$ spack versions boost
and checking the number of processes spawned. Number of processes
per se was not the issue, but each one of them opens a socket
which can stress `iptables`.
In the original issue the kernel watchdog was reporting:
Message from syslogd@login03 at May 19 12:01:30 ...
kernel:Watchdog CPU:110 Hard LOCKUP
Message from syslogd@login03 at May 19 12:01:31 ...
kernel:watchdog: BUG: soft lockup - CPU#110 stuck for 23s! [python3:2756]
Message from syslogd@login03 at May 19 12:01:31 ...
kernel:watchdog: BUG: soft lockup - CPU#94 stuck for 22s! [iptables:5603]
|
|
* Fix parsing of EvtGen URL
* Flake8 fix
|
|
* add an --exclude-file option to 'spack mirror create' which allows a user to specify a file of specs to exclude when creating a mirror. this is anticipated to be useful especially when using the '--all' option
* allow specifying number of versions when mirroring all packages
* when mirroring all specs within an environment, include dependencies of root specs
* add '--exclude-specs' option to allow user to specify that specs should be excluded on the command line
* add test for excluding specs
|