Re:AWS switch from gzip to zstd – about 30% reduction in com...
by natmaka on Sunday August 21, 2022 @06:17AM
(#32538809)
It reacts to the input buffer state (in bad I/O conditions it starves).
On Linux PSI ( /proc/pressure/io ) probably provides more accurate information (the code already uses /proc/cpuinfo ).
Detail: in the fileio.c module there are lines such as:
if (oldIPos == inBuff.pos) inputBlocked++; /* input buffer is full and can't take any more : input speed is faster than consumption rate /
if ( (inputBlocked > inputPresented / 8) / input is waiting often, because input buffers is full : compression or output too slow */
This impacts a 'speedChange' variable.
Its potential values (an enum) are 'noChange', 'slower', and 'faster'.
They are processed rather simply:
if (speedChange == slower) {
((...))
compressionLevel ++;
if (speedChange == faster) {
((...))
compressionLevel --;
On Linux PSI ( /proc/pressure/io ) probably provides more accurate information (the code already uses /proc/cpuinfo ).
Detail: in the fileio.c module there are lines such as:
if (oldIPos == inBuff.pos) inputBlocked++; /* input buffer is full and can't take any more : input speed is faster than consumption rate /
if ( (inputBlocked > inputPresented / 8) / input is waiting often, because input buffers is full : compression or output too slow */
This impacts a 'speedChange' variable.
Its potential values (an enum) are 'noChange', 'slower', and 'faster'. They are processed rather simply: if (speedChange == slower) { ((...)) compressionLevel ++;
if (speedChange == faster) { ((...)) compressionLevel --;