News for hackers. Stuff that matters.
Re:Zstandard – Real-time data compression algorithm by BackBlast on Sunday September 04, 2022 @06:45PM (#32718466)
Technically it is available, but it's just not widely supported. Though it's really unnecessary. Brotli is, in many ways, very comparable or a near equivalent to zstd.
Re:Zstandard – Real-time data compression algorithm by dvhh on Sunday September 04, 2022 @07:17PM (#32718715)
last time I looked Brotli format do not clearly define its frame format, such as there is a good chance that a random stream of data would be decoded as valid brotli compressed data.
Re:Zstandard – Real-time data compression algorithm by BackBlast on Monday September 05, 2022 @06:50AM (#32722862)
When is this ever left to chance? Why would a system randomly decide to try decoding a stream as brotli if it isn't?
Re:Zstandard – Real-time data compression algorithm by Too on Monday September 05, 2022 @11:26AM (#32725343)
Its not so much about suddenly starting to decode something. It’s more when your stream for some reason gets misaligned or corrupted and whatever is still streamed is happily consumed with no possible way to detect the error.

Common reason being someone forgot to check how many bytes read() actually returned and just assume it filled the buffer.

Robust data formats have double protection against this class of bugs with unambiguous SOF and EOF boundaries that the consumer can assert. I guess a sanity-byte once every X Megabytes wouldn’t hurt either.

Re:Zstandard – Real-time data compression algorithm by BackBlast on Tuesday September 06, 2022 @12:04PM (#32739031)
I use brotli as a web distribution format and as an internal compression format on web stacks for various projects and for production systems without issue.

I still don't understand the use case limitation, or exactly where this is a potential issue. Why is the data being interrupted? Is this some kind of raw UDP stream use case?