News for hackers. Stuff that matters.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by hannob on Tuesday May 10, 2022 @10:01AM (#31327221)
It seems somewhat suspicious that the benchmarks don't compare to zstd.

It's not entirely clear to me what the selling point is. "Better than bzip2" isn't exactly a convincing sales pitch given bzip2 is mostly of historic interest these days.

Right now the modern compression field is basically covered by xz (if you mostly care about best compression ratio) and zstd (if you want decent compression and very good speed), so when someone wants to pitch a new compression they should tell me where it stands compared to those.

Re:Bzip3 – A better and stronger spiritual successor to bzip2 by nousermane on Tuesday May 10, 2022 @02:08PM (#31330358)
> benchmarks don't compare to zstd.

  wget http://corpus.canterbury.ac.nz/resources/calgary.tar.gz
  zcat calgary.tar.gz|time zstd -19|wc -c

  902963 (=902.9KB, vs. 807.9KB for bzip3)
> "Better than bzip2" isn't exactly a convincing sales pitch

Sure, but nobody is pitching that. TFA does comapre to lzma (~xz), and claims bzip3 outperforms it quite handsomely in speed, while being competitive in compression ratio.

Re:Bzip3 – A better and stronger spiritual successor to bzip2 by grumpyprole on Tuesday May 10, 2022 @01:24PM (#31329745)
There's more to a compression standard than benchmarks and ratios. Xz does not seem to score well in other, perhaps more important areas: https://www.nongnu.org/lzip/xz_inadequate.html
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by sounds on Tuesday May 10, 2022 @10:31AM (#31327586)
Can you test it out and post results back here?
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by linsomniac on Tuesday May 10, 2022 @12:38PM (#31329142)
For linux-5.17.6.tar:

Original file: 129MB xz, 1.2G uncompressed.

"zstd -T0": 1.34 seconds, 189M

"xz -T0": 63 seconds, 131M

"xz -T0 -9": 183 seconds, 125M

"bzip3 -e -j 6": 21 seconds, 129M (edited, was SIGSEGV)

"bzip3 -e": 84 seconds, 129M

I used linux source because the source website uses linux and recommends bzip3 for compressing source and text. Results were on Ubuntu 22.04, Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz

Re:Bzip3 – A better and stronger spiritual successor to bzip2 by linsomniac on Tuesday May 10, 2022 @01:20PM (#31329699)
One more note:

"bzip3 -e -j 6 -b 50": 25 seconds, 125MB

So nearly as good as the best of xz, but in a 20th the time.

However: Do note that any unexpected use is met with a SIGSEGV: using as a filter, using "-j6" instead of "-j 6", not specifying "-e"...

Re:Bzip3 – A better and stronger spiritual successor to bzip2 by palaiologos on Tuesday May 10, 2022 @01:30PM (#31329824)
lies. not specifying -e displays an error message:

  % bzip3 -e -j 6 -b 50 corpus/calgary.tar
  % bzip3 -j 6 -b 50 corpus/calgary.tar
  bzip3 - A better and stronger spiritual successor to bzip2.
  Copyright (C) by Kamila Szewczyk, 2022. Licensed under the terms of GPLv3.
  Usage: bzip3 [-e/-d/-t/-c] [-b block_size] input output
  Operations:
    -e: encode
    -d: decode
    -t: test
  Extra flags:
    -c: force reading/writing from standard streams
    -b N: set block size in MiB
    -j N: set the amount of parallel threads
you can use bzip3 as a filter:

  % cat corpus/calgary.tar | bzip3 -b 10 -e -c | wc -c
  807959
and using "-j6" is simply being unable to read the help page.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by eesmith on Tuesday May 10, 2022 @02:54PM (#31331039)
The code has, at https://github.com/kspalaiologos/bzip3/blob/bf2f0e02fd59f4c4... :

            } else if (argv[i][1] == 'j') {
                workers = atoi(argv[i + 1]);
                i++;
If the last argument is "-j6" then this will read past the end of the allocated argv strings and try to do atoi(NULL):

  % ./bzip3 -j3 < README.md
  Segmentation fault
"-j6" is standard getopt() behavior, and the default expected behavior from Unix/POSIX systems.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by ttybird2 on Tuesday May 10, 2022 @03:03PM (#31331176)
They are not disputing that - they actually acknowledged it. Instead they are disputing that omitting -e will lead to a crash.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by linsomniac on Wednesday May 11, 2022 @09:12AM (#31339443)
You're right, what I thought was encoding/filter SEGVs were the bug in handling "-j6".
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by JoshTriplett on Tuesday May 10, 2022 @01:35PM (#31329888)
If you ramp up the compression level on zstd, does it get smaller than bzip3 before it gets to the point of taking more time?
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by nousermane on Tuesday May 10, 2022 @03:18PM (#31331362)
No, not even if you disregard time altogether. Compression of Calgary Corpus on a random old laptop:

             sec   KB
  gzip      0.19  1070
  zstd      0.02  1063
  zstd -19  1.47   897
  bzip2     0.35   891
  brotli    8.12   862
  xz        1.35   853
  bzip3     0.52   808
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by palaiologos on Tuesday May 10, 2022 @12:49PM (#31329327)
it's `bzip3 -e -j 6`. you need a space.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by adgjlsfhk1 on Tuesday May 10, 2022 @01:54PM (#31330155)
Can you also give decompression speeds?
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by linsomniac on Wednesday May 11, 2022 @09:23AM (#31339591)
"zstd -d": 1.03 seconds

"xz -d": 7.92 seconds

"bzip3 -d" as filter: SEGSEGV

"bzip3 -d linux-5.17.6.tar.bz3": 81 seconds

"bzip -d -j 6": 21.35 seconds (edit)

Lest I be called a liar again:

  $ ./bzip3-1.0.1/bzip3 -d <linux-5.17.6.tar.bz3 >/dev/null
  fish: Job 1, 'time ./bzip3-1.0.1/bzip3 -d <li…' terminated by signal SIGSEGV (Address boundary error)
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by palaiologos on Wednesday May 11, 2022 @11:04AM (#31341015)
Consider using `-c`, which makes the compressor use standard streams, or pull the main branch because I had just pushed a tiny patch that automatically enables it when no positional arguments are given.
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by Matumio on Tuesday May 10, 2022 @11:45AM (#31328481)
No, but it would be nice to see visually where it is on the size vs (de-)compression speed pareto-front. Like this graphic (from the zstd homepage): https://raw.githubusercontent.com/facebook/zstd/master/doc/i...
Re:Bzip3 – A better and stronger spiritual successor to bzip2 by codewiz on Tuesday May 10, 2022 @01:50PM (#31330100)
This chart is from 2016. Both zstd and brotli are under active development, so I'd like to see a more recent comparison presented in this same format.