I've reached the same conclusion, and I've been using zstd unless I want extremely fast (de)compression with lz4.

And even when I still have to use gzip (the format), the executables of libdeflate are fully compatible and noticeably faster than gzip/gunzip (the tool).

This was on HN a while ago https://jolynch.github.io/posts/use_fast_data_algorithms/ and reaches the same conclusion. I am inclined to use Zstd even in cases where people would use lz4. Zstd is very fast.

Do zless, zcat and zgrep support zstd everywhere? And I mean everywhere? VMs? Alpine? FreeBSD? OSX? Openwrt?

Nothing is shittier than sshing into a box that doesn’t understand half of your command line tricks. Or the clever shell script you just tested six ways to Sunday. It’s like fighting with your hands tied behind your back.

Yes, it's been integrated in lots of places, mostly as long ago as the late 2010s.

> other than backwards compatibility

Other than that, Mrs Lincoln, how was the play?

Yes, it's supported on all those platforms.

How far back?

How much does that matter? All currently supported releases at least.

Seriously?

There are places still migrating off of Java 8.

And pet servers still exist in the wild.

Seriously who cares about those who are living in the past? It is on them for not keeping up with the times.

Java 8 will be supported until 2032 if you pick the right distro, and that distro comes with zstd in the repos (if not pre-installed).

And if your system is so old that you don't have access to eight-year-old software, I don't see why the rest of the world needs to accommodate you. Sticking with weird and old software is fine, but that comes with the risk of having to deal with everyone else moving on.

bash4 was released Feb 2009. 16 years ago. The very latest version of osx ships with bash3, from 20 years ago.

It's a fair question, and "currently supported" is essentially meaningless.

(Yes we all know about macports and the other more popular but less correct systems, and yes most bash scripts should be written to specifically avoid relying on any special features exactly to avoid being broken so easily. These are both true and both beside the point. Fact is still, this is not only a "currently supported" system by some technicality like the final months of a 5 year lts release or something, this is the up to this very minute version of a massive install base, with a 20 year old version of something as basic and all-touching as the very shell itself.

I know about this so intimately because I actually have scripts that don't work because they were intentional stunts to see just what could be done without forking any external exes or even child shells, but allow using every possible feature and trick in bash itself.)

The bash 4 thing is due to the GPL 3, not some inherent slowness in updating software. It has nothing to do with zstd, which is permissively licensed.

It doesn't matter why. All that matters is that "current" is not a valid word. Old things exist in "current" systems. And current systems may also be old systems.

Ok. I think engineers are well capable of evaluating what systems they need to support and if zstd is a usable option for them. In many situations, the answer will be "yes."

You can download a supported zstd release through homebrew for every version of macOS that is supported by Apple (as those are the releases homebrew supports), and it compiles from source for older ones.

I don't even know why we're talking about bash here. But for the record, this is a unique circumstance that affects only software which made the transition from GPLv2 -> GPLv3. Changing your license can cause you to fork away from some users. News at 11.

I really don’t want to switch to zsh, but these are facts.

The number of devs working on OSX producing non-OSX applications is staggering.

What does that have to do with zstd?

It directly addresses "How much does that matter? All currently supported releases at least."

No it doesn’t? You can get zstd on macOS just fine. We’re talking about zstd, not bash, and supported releases not what ships with the distribution.

You can get bash4 or 5 too, as I already said. I can't help you with your depth of experience. I didn't even say zstd was unfit to depend on, just that it was a fair question.

Once again, I’m having to remind a person that coding is a team sport and personal choices are bullshit.

Every deviation you make from stock is something else you have to put into. The onboarding docs and convince all your coworkers to install. They won’t. Or they think they will but will miss a step or machine. Or for language specific tools, they’ll use a multiversion tool like asdf or nvm or ram and have to reinstall again every upgrade.

It’s not up to me. It’s not up to you. It’s up to us. And if you can’t see that, maybe you shouldn’t be working with other developers. Or maybe you haven’t been and you need to.

zstd is a great setup, for sure, but the build system they use is patently awful. Can someone make an autoconf or hell, even CMake build system for them pleasee???

Just install the binary package from whatever distro you use? Why do you need to build it?

But if it matters, FreeBSD has a pretty trivial BSDmake build of it:

https://github.com/freebsd/freebsd-src/blob/main/lib/libzstd...

https://github.com/freebsd/freebsd-src/blob/main/usr.bin/zst...

You could easily do something similar in GNU make or whatever without the dependencies on the FBSD build system. It's basically just the classic "cc -c a bunch of C files, then link them."

I suppose the point of open source software is to edit the source code and make your own version. If you can't build it, you can't do that.

Zstd has lots of contributors who have all figured out how to edit and build it.

Yeah, not sure what the original commenter is complaining about. I did "git clone https://github.com/facebook/zstd", "cd zstd", "make" and have a working binary. Doesn't get easier than that.

Not everyone is using a standard tool chain or anything like that and oftentimes when I find projects with atypical build systems that give me trouble I tend to just move on to something else

It's basically a giant GCCism. Don't be condescending.

I have no idea what you're talking about at this point.

> Just install the binary package from whatever distro you use? Why do you need to build it?

Cuz I'm actually a packaging maintainer for a couple different old operating systems. Regardless I thank you for being thankful even though I feel like you were being backhanded and demeaning on a number of levels.

> you were being backhanded and demeaning on a number of levels.

What an awful comment to make.

Quick glance of zstd github repo shows they do provide CMake build scripts?

Last I touched it they only had glitchy make files and I don't like cmake particularly, I was just like "bespoke make files are fine if they're posix compliant, but these aren't"

> There is no reason to use gzip these days other than backwards compatibility

And forward compatibility. The Lindy effect says gzip is likelier to be widely available across platforms and tools in the long term than zstd.

Zip beats all on that front.

Yep!

> bzip2 -9 -k all.tar 981.78s user 9.77s system 95% cpu 8M memory 17:16.64 total

> bzip3 -e -b 256 -j 12 all.tar 2713.81s user 16.28s system 634% cpu 18301M memory 7:10.10 total

The memory usage is one reason: 8M vs 18301M

I think you intended to reply somewhere else. This isn't responsive.

yes - thanks!