I found an use for lossless streaming:

A weakness in the open source landscape I found is that there's a lack of fast codecs. If you want to encode in real time (eg, streaming speech or an artist playing in the moment), and encode per client for positional audio in a 3D world, then Opus becomes a bottleneck.

FLAC according to some preliminary tests helps by being faster than Opus to encode.

Both FLAC and Opus[0][1] have a dial to trade efficiency against CPU time. Have you even tried? I doubt FLAC remains a superior choice at low-complexity Opus settings. The library also has a complexity setting for the encoder.

[0] https://opus-codec.org/docs/opus-tools/opusenc.html [1] > Set encoding computational complexity (0–10, default: 10). Zero gives the fastest encodes but lower quality, while 10 gives the highest quality but slower encoding.

I have tried it. It works, but the difference in my experience isn't a huge one.

Complexity 0 is about twice as fast as 10. Which isn't bad, but if you want to go really fast it still leaves something to be desired.

I hope this has been measured with the binary optimized for the CPU architecture in question, in oder to sqeeze out reasonable gains from likely more portability-focused opus code, particularly regarding vectorization opportunities.

By "fast to encode" do you mean that in terms of low latency or low CPU usage?

I ask because I know Opus is used quite commonly in real-time applications (e.g. voip), and I remember when researching in the past that it is actually capable of lower inherent latency because of support for smaller frame sizes than some of its competition.

I haven't looked into how expensive it is to encode in terms of CPU time, so I assume maybe you're taking about a bottleneck in terms of the number of simultaneous streams you can support on a single CPU?

Both, primarily the first. Low latency is definitely a requirement, but no issues with Opus on that account.

> I haven't looked into how expensive it is to encode in terms of CPU time, so I assume maybe you're taking about a bottleneck in terms of the number of simultaneous streams you can support on a single CPU?

Yup! I work on https://vircadia.com/ -- we have to compress audio in real time and every user gets their own mix since it depends on their location in 3D space. It turns out to add up pretty fast, and you can't fit that many people into a cheap VPS.

That's why I'm working on FLAC support for it. If bandwidth is plentiful but CPU resources are lacking it's a good alternative to have.

This code originally came from the High Fidelity company, which made their own codec. It's some piece of black magic that cuts down audio by exactly 1/4th and is amazingly fast. But it's closed source.

There's no way their proprietary codec is less black magic than Opus is. I've rarely if ever seen a closed source codec designed by people who really know what they're doing; they're usually just whatever the cowboy they hired thought would be cool. It's not like management is going to be able to tell if it's state of the art or not.

Well, I say "black magic" because at least to me it's a quite mysterious creation. But I'm no audio engineer.

It's a lossy codec that's for some reason fixed rate, and shrinks audio by exactly 1/4th. The resulting quality is very good (works perfectly fine for music), and it's much faster than Opus. It retains stereo and high frequencies and sounds just fine. I'm sure there's a tradeoff somewhere, but it's certainly good enough to hold music events without people complaining.

I think it might be possibly related to codecs like AptX.

So far I've not found anything competitive that we could use -- stuff is either closed, or far more CPU intensive.

Yeah, it sounds like ADPCM with a fixed ratio like that. It's extremely simple and adds no latency, which are both good for streaming.

Usually a fixed compression ratio is a terrible tradeoff, but for streaming it kind-of makes sense.

I don't think it's ADPCM since it sounds really good. I'm not an audiophile or have done any comprehensive testing, but it' the kind of quality a normal person would use for music without complaining.

That was meant to agree with you - AptX is a kind of ADPCM with some other filters added on, so this could be the same thing. That's how it gets low latency.

What are the use-cases for pre-baked positional audio? Ie where the client can't do the positional audio processing itself.

Not pre-baked. I work on https://vircadia.com/ -- it has to give every user their own audio mix to account for their 3D position.

This means codec costs add up fast. You can't just encode once and stream the same thing to a dozen people.

Oh right, I read that wrong. Thought you did per-voice 3D positional encoding on server but mixing on client.

Interesting use-case, got my brain racing off to find silly solutions.