News for hackers. Stuff that matters.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by zigzag312 on Thursday December 23, 2021 @02:04PM (#29665219)
Is there any open source audio compression format like that? Lossless and very fast. I haven't found any yet.

EDIT: I'm thinking about a format that would be suitable as a replacement for uncompressed WAV files in DAWs. Rendered tracks often have large sections of silence and uncompressed WAVs have always seemed wasteful to me.

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by LeoPanthera on Thursday December 23, 2021 @02:24PM (#29665447)
FLAC is always lossless, but has a variable compression ratio so you can trade compression for speed.

Using the command line "flac" tool, "flac -0" is the fastest, "flac -8" is the slowest, but produces the smallest files.

In my experience, 0-2 all produce roughly equivalent sized files, as do 4-8.

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by makapuf on Thursday December 23, 2021 @04:24PM (#29666647)
I tried passing stereo wavs in 2 x 16bits (4bytes) as rgba for qoi but I haven't been very successful.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by adgjlsfhk1 on Thursday December 23, 2021 @05:36PM (#29667288)
That's not surprising. QOI is heavily optimized for images which tend to be relatively continuous, while audio tends to oscillate a ton.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by cycomanic on Friday December 24, 2021 @02:36AM (#29671278)
It might work to fourier transform first (although likely will kill performance)
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by adgjlsfhk1 on Friday December 24, 2021 @02:41AM (#29671320)
fixed size FFT (eg length 64) can be made scary fast.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by wombatmobile on Thursday December 23, 2021 @04:39PM (#29666813)
I'd also like to know what's the best (or any) lossless audio compression process/tools.

My application is to send audio (podcast recordings) to a remote audio engineer friend who will do the post processing, then round trip it to me to complete the editing.

Wav is so big it makes a 1 hr podcast a difficult proposition.

MP3 is unsuitable because compression introduces too many artefacts the quality suffers unacceptably.

What do other people do in this circumstances?

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by phonon on Thursday December 23, 2021 @05:49PM (#29667408)
1 hour of CD quality mono FLAC encoded is about 100-150 MB. Is that small enough?
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by wombatmobile on Thursday December 23, 2021 @07:42PM (#29668481)
Well, I'm using a Rodecaster in multi-channel mode with 3 mics so an hour is more like 450MB.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by selectodude on Thursday December 23, 2021 @05:06PM (#29667047)
FLAC and ALAC can be losslessly converted to back to WAV and cuts the file size in half.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by StreamBright on Thursday December 23, 2021 @02:06PM (#29665247)
ALAC? FLAC? What is the problem with these?
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by zigzag312 on Thursday December 23, 2021 @02:28PM (#29665502)
FLAC is limited to 24 bit depth. I was thinking of intermediate format suitable for use in DAWs and samplers that also supports floating point to avoid clipping.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by LeoPanthera on Thursday December 23, 2021 @02:31PM (#29665549)
24-bit integer and 32-bit float have the same dynamic range available, so you are not losing any fidelity.

However, frankly, if you're working professionally with audio like that, the best solution is simply to have sufficient disk space available to work with raw audio.

Use FLAC to compress the final product, when you are done.

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by kloch on Thursday December 23, 2021 @03:07PM (#29665883)
They have the same precision but float has vastly larger dynamic range due to the 8-bit exponent. When normalized and quantized for output this does result in roughly the same effective dynamic range (depending on how much of the integer range was originally used).

The issue is audio is typically mixed close to maximum so any processing steps can easily lead to clipping. One solution is to use float or larger integers internally during each processing step and normalize/convert back to 24-bit integer to write to disk. Another (better imo) option would be to do all intermediate steps and disk saves in a floating point format and only normalize/quantize for output once.

I haven't worked with professional audio in over 25 years (before everything went fully digital) but I would be surprised if floating point formats were not an option for encoding and intermediate workflows. Many quantization steps seems like a bad idea.

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by adzm on Thursday December 23, 2021 @03:17PM (#29666001)
Most DAWs and plugins and audio interfaces nowadays use floating point internally.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by zigzag312 on Thursday December 23, 2021 @03:31PM (#29666114)
> I would be surprised if floating point formats were not an option for encoding and intermediate workflows.

For bouncing tracks to disk, uncompressed 32-bit floating point formats are avaliable, but I am not aware of any fast losslessly compressed 32-bit floating point format.

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by 323 on Thursday December 23, 2021 @03:42PM (#29666226)
All professional audio production software these days internally works with 32/64 bit floats. That's the native format, because it allows you to go above 0 dBFS (maximum level), as long as you go back below it at the end of the chain.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by zigzag312 on Thursday December 23, 2021 @02:38PM (#29665623)
With 24-bit integer you are at risk of clipping.

EDIT: Floating point is useful while you are working to avoid any accidental clipping. As an intermediate format, like a ProRes for video. FLAC is great as a final format.

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by artiii on Thursday December 23, 2021 @03:15PM (#29665980)
check WavPack (32pcm, floats etc) but it's slower(not much) than flac, offering slighty beter compresion.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by zigzag312 on Thursday December 23, 2021 @03:43PM (#29666236)
WavPack seems a bit too slow already. 3x slower decode compared to FLAC in this test https://stsaz.github.io/fmedia/audio-formats/
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by TylerE on Thursday December 23, 2021 @09:54PM (#29669517)
Wavpack on a modern CPU, from your own link, decodes at approx. 250x realtime. How fast is 'fast enough' if that isn't?
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by zigzag312 on Friday December 24, 2021 @02:40AM (#29671307)
Projects with 100+ tracks are not uncommon. Sampler/rompler of a single virtual instrument can play 10+ sounds simultaneously. Playback of an orchestral score with virtual instruments can easily go over 250 simultaneous sounds, so just a real-time playback (without any additional processing) would already be a challenge.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by TylerE on Friday December 24, 2021 @04:23AM (#29671944)
In those contexts samples are loaded once and then kept in RAM.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by zigzag312 on Friday December 24, 2021 @05:12AM (#29672290)
No, actually only first few dozen KB of each sample are usually preloaded into RAM. The rest is streamed from a SSD. One library of an orchestral section can have 100+ GB of samples. You wouldn't fit all sections in 128GB of RAM.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by TylerE on Friday December 24, 2021 @05:16AM (#29672309)
I'm quite aware of the size of large libraries, I own a number of them.

Part of why I built my current machine with 64GB of ram

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by StreamBright on Thursday December 23, 2021 @05:47PM (#29667385)
Nice, I was not aware.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by 323 on Thursday December 23, 2021 @03:36PM (#29666166)
WavPack might fit the bill. It has decent software support. Not sure if DAWs can use it natively, they might unpack it to a temp folder.

https://www.wavpack.com/

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by zigzag312 on Thursday December 23, 2021 @03:47PM (#29666283)
Reaper does. Unfortunately, WavPack has a bit too much performance overhead.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by 323 on Thursday December 23, 2021 @03:53PM (#29666336)
It's a 20 year old format.

ZStandard is a very good compressor, with an especially fast decompressor. Maybe someone should try using this instead of zlib in an audio format (FLAC, WavPack, ...)

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by BlueSwordM on Thursday December 23, 2021 @05:02PM (#29667017)
I mean, is there really a need for utilizing ZStd for audio compression?

FLAC is extremely good at compression audio, has very fast encode and uber fast decode. It also doesn't use zlib...

Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by dmitrygr on Thursday December 23, 2021 @02:05PM (#29665236)
gzip -1 is lossless and fast. It will somewhat compress pcm data :)
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by zigzag312 on Thursday December 23, 2021 @02:30PM (#29665530)
You would loose fast seeking ability with gzip. Or am I mistaken?
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by jzwinck on Thursday December 23, 2021 @04:18PM (#29666591)
You can only seek within a gzip file if you write it with some number of Z_FULL_FLUSH points which are resumable. The command line gzip program does not support this, but it's easy using zlib. For example you might do a Z_FULL_FLUSH roughly every 50 MB of compressed data. Then you can seek to any byte in the file, search forward or backward for the flush marker, and decompress forward from there as much as you want. If your data is sorted or is a time series, it's easy to implement binary search this way. And standard gunzip etc will still be able to read the whole file as normal.
Re:QOI – The “Quite OK Image Format” for fast, lossless imag... by flohofwoe on Friday December 24, 2021 @05:34AM (#29672418)
MOD files ;)

(but seriously, MODs can encode hours of audio into kilobytes, the downside is of course that they require a special authoring process which seems to be a bit of a lost art today)