Re:Tape still beats SSDs and hard drives when it comes to pr...
by nightfly on Wednesday September 25, 2019 @02:25AM
(#21068225)
pigz!
[ Parent
| on HN ]
Re:Tape still beats SSDs and hard drives when it comes to pr...
by terrelln on Wednesday September 25, 2019 @06:41PM
(#21076385)
Zstd also has a multithreaded mode. On the CLI `zstd -T0` tells zstd to use all available cores.
Re:Tape still beats SSDs and hard drives when it comes to pr...
by namibj on Thursday September 26, 2019 @07:36AM
(#21079958)
Please be aware that this will significantly impact compression ratio. So much actually, that I don't dare use it by default. I'd often use the non-multithreaded mode as for some data (perf.data files with large stack captures, most recently) and reasonable compression speeds of >=50MB/s input multi-threaded modes are not pareto-optimal.
I'll get a small benchmark script for that done over lunch. Come back here later.
Re:Tape still beats SSDs and hard drives when it comes to pr...
by Cyan4973 on Thursday September 26, 2019 @06:11PM
(#21086132)
> Please be aware that this will significantly impact compression ratio.
Is this a continuation from previous message ? Because multithreading mode of zstd doesn't impact its compression ratio. It's actually an interesting property of zstd : whatever the nb of threads used, the compression ratio remains the same (aka. reproducible).