diff options
author | Tom Scogland <scogland1@llnl.gov> | 2021-05-31 20:33:14 -0700 |
---|---|---|
committer | GitHub <noreply@github.com> | 2021-05-31 20:33:14 -0700 |
commit | 4a7b0afde2e04305697da2d441a560c2f976c3c4 (patch) | |
tree | 061d3312e044cdd85efbf64ab70cba7d35b9b62e /README.md | |
parent | ea4a2c9120d86da2e647c348e00c7cc477e37bd1 (diff) | |
download | spack-4a7b0afde2e04305697da2d441a560c2f976c3c4.tar.gz spack-4a7b0afde2e04305697da2d441a560c2f976c3c4.tar.bz2 spack-4a7b0afde2e04305697da2d441a560c2f976c3c4.tar.xz spack-4a7b0afde2e04305697da2d441a560c2f976c3c4.zip |
Log performance improvement (#23925)
* util.tty.log: read up to 100 lines if ready
Rework to read up to 100 lines from the captured stdin as long as data
is ready to be read immediately. Adds a helper function to poll with
`select` for ready data. This showed a roughly 5-10x perf improvement
for high-rate writes through the logger with relatively short lines.
* util.tty.log: Defer flushes to end of ready reads
Rather than flush per line, flush per set of reads. Since this is a
non-blocking loop, the total perceived wait is short.
* util.tty.log: only scan each line once, usually
Rather than always find all control characters then substitute them all,
use `subn` to count the number of control characters replaced. Only if
control characters exist find out what they are. This could be made
truly single pass with sub with a function, but it's a more intrusive
change and this got 99%ish of the performance improvement (roughly
another 2x in some cases).
* util.tty.log: remove check for `readable`
Python < 3 does not support a readable check on streams, should not be
necessary here since we control the only use and it's explicitly a
stream to be read.
Diffstat (limited to 'README.md')
0 files changed, 0 insertions, 0 deletions