Re:Bzip3 – A better and stronger spiritual successor to bzip2
by linsomniac on Tuesday May 10, 2022 @01:20PM
(#31329699)
One more note:
"bzip3 -e -j 6 -b 50": 25 seconds, 125MB
So nearly as good as the best of xz, but in a 20th the time.
However: Do note that any unexpected use is met with a SIGSEGV: using as a filter, using "-j6" instead of "-j 6", not specifying "-e"...
Re:Bzip3 – A better and stronger spiritual successor to bzip2
by palaiologos on Tuesday May 10, 2022 @01:30PM
(#31329824)
lies. not specifying -e displays an error message:
% bzip3 -e -j 6 -b 50 corpus/calgary.tar
% bzip3 -j 6 -b 50 corpus/calgary.tar
bzip3 - A better and stronger spiritual successor to bzip2.
Copyright (C) by Kamila Szewczyk, 2022. Licensed under the terms of GPLv3.
Usage: bzip3 [-e/-d/-t/-c] [-b block_size] input output
Operations:
-e: encode
-d: decode
-t: test
Extra flags:
-c: force reading/writing from standard streams
-b N: set block size in MiB
-j N: set the amount of parallel threads
you can use bzip3 as a filter: % cat corpus/calgary.tar | bzip3 -b 10 -e -c | wc -c
807959
and using "-j6" is simply being unable to read the help page.
Re:Bzip3 – A better and stronger spiritual successor to bzip2
by eesmith on Tuesday May 10, 2022 @02:54PM
(#31331039)
The code has, at https://github.com/kspalaiologos/bzip3/blob/bf2f0e02fd59f4c4... :
} else if (argv[i][1] == 'j') {
workers = atoi(argv[i + 1]);
i++;
If the last argument is "-j6" then this will read past the end of the allocated argv strings and try to do atoi(NULL): % ./bzip3 -j3 < README.md
Segmentation fault
"-j6" is standard getopt() behavior, and the default expected behavior from Unix/POSIX systems.
Re:Bzip3 – A better and stronger spiritual successor to bzip2
by ttybird2 on Tuesday May 10, 2022 @03:03PM
(#31331176)
They are not disputing that - they actually acknowledged it. Instead they are disputing that omitting -e will lead to a crash.
Re:Bzip3 – A better and stronger spiritual successor to bzip2
by linsomniac on Wednesday May 11, 2022 @09:12AM
(#31339443)
You're right, what I thought was encoding/filter SEGVs were the bug in handling "-j6".
Re:Bzip3 – A better and stronger spiritual successor to bzip2
by JoshTriplett on Tuesday May 10, 2022 @01:35PM
(#31329888)
If you ramp up the compression level on zstd, does it get smaller than bzip3 before it gets to the point of taking more time?
Re:Bzip3 – A better and stronger spiritual successor to bzip2
by nousermane on Tuesday May 10, 2022 @03:18PM
(#31331362)
No, not even if you disregard time altogether. Compression of Calgary Corpus on a random old laptop:
sec KB
gzip 0.19 1070
zstd 0.02 1063
zstd -19 1.47 897
bzip2 0.35 891
brotli 8.12 862
xz 1.35 853
bzip3 0.52 808
Re:Bzip3 – A better and stronger spiritual successor to bzip2
by palaiologos on Tuesday May 10, 2022 @12:49PM
(#31329327)
it's `bzip3 -e -j 6`. you need a space.
Re:Bzip3 – A better and stronger spiritual successor to bzip2
by adgjlsfhk1 on Tuesday May 10, 2022 @01:54PM
(#31330155)
Can you also give decompression speeds?
Re:Bzip3 – A better and stronger spiritual successor to bzip2
by linsomniac on Wednesday May 11, 2022 @09:23AM
(#31339591)
"zstd -d": 1.03 seconds
"xz -d": 7.92 seconds
"bzip3 -d" as filter: SEGSEGV
"bzip3 -d linux-5.17.6.tar.bz3": 81 seconds
"bzip -d -j 6": 21.35 seconds (edit)
Lest I be called a liar again:
$ ./bzip3-1.0.1/bzip3 -d <linux-5.17.6.tar.bz3 >/dev/null
fish: Job 1, 'time ./bzip3-1.0.1/bzip3 -d <li…' terminated by signal SIGSEGV (Address boundary error)
Re:Bzip3 – A better and stronger spiritual successor to bzip2
by palaiologos on Wednesday May 11, 2022 @11:04AM
(#31341015)
Consider using `-c`, which makes the compressor use standard streams, or pull the main branch because I had just pushed a tiny patch that automatically enables it when no positional arguments are given.
Original file: 129MB xz, 1.2G uncompressed.
"zstd -T0": 1.34 seconds, 189M
"xz -T0": 63 seconds, 131M
"xz -T0 -9": 183 seconds, 125M
"bzip3 -e -j 6": 21 seconds, 129M (edited, was SIGSEGV)
"bzip3 -e": 84 seconds, 129M
I used linux source because the source website uses linux and recommends bzip3 for compressing source and text. Results were on Ubuntu 22.04, Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz