diff options
author | Todd Gamblin <gamblin2@llnl.gov> | 2021-08-24 14:08:34 -0700 |
---|---|---|
committer | GitHub <noreply@github.com> | 2021-08-24 14:08:34 -0700 |
commit | 1374fea5d9f5dc565141086b4fe70adb7b19f5f2 (patch) | |
tree | 7f0b2cb5173dbe89c5d5971e5540cfc0dba756ec /bin | |
parent | 7274d8bca2b0cb72e57fb24eb9df843be97c3aa0 (diff) | |
download | spack-1374fea5d9f5dc565141086b4fe70adb7b19f5f2.tar.gz spack-1374fea5d9f5dc565141086b4fe70adb7b19f5f2.tar.bz2 spack-1374fea5d9f5dc565141086b4fe70adb7b19f5f2.tar.xz spack-1374fea5d9f5dc565141086b4fe70adb7b19f5f2.zip |
locks: only open lockfiles once instead of for every lock held (#24794)
This adds lockfile tracking to Spack's lock mechanism, so that we ensure that there
is only one open file descriptor per inode.
The `fcntl` locks that Spack uses are associated with an inode and a process.
This is convenient, because if a process exits, it releases its locks.
Unfortunately, this also means that if you close a file, *all* locks associated
with that file's inode are released, regardless of whether the process has any
other open file descriptors on it.
Because of this, we need to track open lock files so that we only close them when
a process no longer needs them. We do this by tracking each lockfile by its
inode and process id. This has several nice properties:
1. Tracking by pid ensures that, if we fork, we don't inadvertently track the parent
process's lockfiles. `fcntl` locks are not inherited across forks, so we'll
just track new lockfiles in the child.
2. Tracking by inode ensures that referencs are counted per inode, and that we don't
inadvertently close a file whose inode still has open locks.
3. Tracking by both pid and inode ensures that we only open lockfiles the minimum
number of times necessary for the locks we have.
Note: as mentioned elsewhere, these locks aren't thread safe -- they're designed to
work in Python and assume the GIL.
Tasks:
- [x] Introduce an `OpenFileTracker` class to track open file descriptors by inode.
- [x] Reference-count open file descriptors and only close them if they're no longer
needed (this avoids inadvertently releasing locks that should not be released).
Diffstat (limited to 'bin')
0 files changed, 0 insertions, 0 deletions