News for hackers. Stuff that matters.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by linsomniac on Tuesday May 10, 2022 @12:38PM (#31329142)
For linux-5.17.6.tar:

Original file: 129MB xz, 1.2G uncompressed.

"zstd -T0": 1.34 seconds, 189M

"xz -T0": 63 seconds, 131M

"xz -T0 -9": 183 seconds, 125M

"bzip3 -e -j 6": 21 seconds, 129M (edited, was SIGSEGV)

"bzip3 -e": 84 seconds, 129M

I used linux source because the source website uses linux and recommends bzip3 for compressing source and text. Results were on Ubuntu 22.04, Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz

Re:Bzip3 – A better and stronger spiritual successor to bzip2 by linsomniac on Tuesday May 10, 2022 @01:20PM (#31329699)
One more note:

"bzip3 -e -j 6 -b 50": 25 seconds, 125MB

So nearly as good as the best of xz, but in a 20th the time.

However: Do note that any unexpected use is met with a SIGSEGV: using as a filter, using "-j6" instead of "-j 6", not specifying "-e"...

Re:Bzip3 – A better and stronger spiritual successor to bzip2 by palaiologos on Tuesday May 10, 2022 @01:30PM (#31329824)
lies. not specifying -e displays an error message:

  % bzip3 -e -j 6 -b 50 corpus/calgary.tar
  % bzip3 -j 6 -b 50 corpus/calgary.tar
  bzip3 - A better and stronger spiritual successor to bzip2.
  Copyright (C) by Kamila Szewczyk, 2022. Licensed under the terms of GPLv3.
  Usage: bzip3 [-e/-d/-t/-c] [-b block_size] input output
  Operations:
    -e: encode
    -d: decode
    -t: test
  Extra flags:
    -c: force reading/writing from standard streams
    -b N: set block size in MiB
    -j N: set the amount of parallel threads
you can use bzip3 as a filter:

  % cat corpus/calgary.tar | bzip3 -b 10 -e -c | wc -c
  807959
and using "-j6" is simply being unable to read the help page.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by eesmith on Tuesday May 10, 2022 @02:54PM (#31331039)
The code has, at https://github.com/kspalaiologos/bzip3/blob/bf2f0e02fd59f4c4... :

            } else if (argv[i][1] == 'j') {
                workers = atoi(argv[i + 1]);
                i++;
If the last argument is "-j6" then this will read past the end of the allocated argv strings and try to do atoi(NULL):

  % ./bzip3 -j3 < README.md
  Segmentation fault
"-j6" is standard getopt() behavior, and the default expected behavior from Unix/POSIX systems.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by ttybird2 on Tuesday May 10, 2022 @03:03PM (#31331176)
They are not disputing that - they actually acknowledged it. Instead they are disputing that omitting -e will lead to a crash.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by linsomniac on Wednesday May 11, 2022 @09:12AM (#31339443)
You're right, what I thought was encoding/filter SEGVs were the bug in handling "-j6".
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by JoshTriplett on Tuesday May 10, 2022 @01:35PM (#31329888)
If you ramp up the compression level on zstd, does it get smaller than bzip3 before it gets to the point of taking more time?
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by nousermane on Tuesday May 10, 2022 @03:18PM (#31331362)
No, not even if you disregard time altogether. Compression of Calgary Corpus on a random old laptop:

             sec   KB
  gzip      0.19  1070
  zstd      0.02  1063
  zstd -19  1.47   897
  bzip2     0.35   891
  brotli    8.12   862
  xz        1.35   853
  bzip3     0.52   808
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by palaiologos on Tuesday May 10, 2022 @12:49PM (#31329327)
it's `bzip3 -e -j 6`. you need a space.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by adgjlsfhk1 on Tuesday May 10, 2022 @01:54PM (#31330155)
Can you also give decompression speeds?
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by linsomniac on Wednesday May 11, 2022 @09:23AM (#31339591)
"zstd -d": 1.03 seconds

"xz -d": 7.92 seconds

"bzip3 -d" as filter: SEGSEGV

"bzip3 -d linux-5.17.6.tar.bz3": 81 seconds

"bzip -d -j 6": 21.35 seconds (edit)

Lest I be called a liar again:

  $ ./bzip3-1.0.1/bzip3 -d <linux-5.17.6.tar.bz3 >/dev/null
  fish: Job 1, 'time ./bzip3-1.0.1/bzip3 -d <li…' terminated by signal SIGSEGV (Address boundary error)
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by palaiologos on Wednesday May 11, 2022 @11:04AM (#31341015)
Consider using `-c`, which makes the compressor use standard streams, or pull the main branch because I had just pushed a tiny patch that automatically enables it when no positional arguments are given.