Re:SoundStream: An End-to-End Neural Audio Codec
by johntb86 on Sunday August 15, 2021 @03:43AM
(#28186928)
Has anyone thought of trying to end-to-end train an h.265 decoder? The results might not be 100% perfect, but the resulting codec might bypass a ton of patents.
[ Parent
| on HN ]
Re:SoundStream: An End-to-End Neural Audio Codec
by vletal on Sunday August 15, 2021 @03:59AM
(#28186981)
Since the h265 relies heavily on operations which are not easily differentiable, such as translation of patches of images, together with a pretty complicated binary format, I'd be pretty amazed if the NN actually learned anything meaningful at all.
Re:SoundStream: An End-to-End Neural Audio Codec
by svantana on Sunday August 15, 2021 @06:06AM
(#28187585)
Interpolated translation is continuous and easily differentiated. There's lots of work on machine-learned video codecs already, from Nvidia, Qualcomm and others.
Re:SoundStream: An End-to-End Neural Audio Codec
by zinekeller on Sunday August 15, 2021 @07:38AM
(#28187941)
> There's lots of work on machine-learned video codecs already, from Nvidia, Qualcomm and others.
I've tried them, the current state of the art means that they're only useful on relatively static things (some shaking etc) while spike up to AV1-level bitrate to reach perceptual similarity when the movement is too much. Maybe in the future (or whatever under the wraps concoction Nvidia, Qualcomm or another player have), ML-based video codecs will surpass handtuned codecs, but it's not (yet) the present state.
Re:SoundStream: An End-to-End Neural Audio Codec
by vletal on Sunday August 15, 2021 @06:08AM
(#28187594)
It's not difficult to propagate gradients while translating an image. Learning "pick a 8x8 patch from (145,17) apply X to it and translate it by (4,-8)" from data is on different level, is not it?
The premise was using e2e learning to avoid patent issues. I am sure that with some preprocessing you can plug a NN inside the deciding process and learn very meaningful stuff.
Re:SoundStream: An End-to-End Neural Audio Codec
by itsamesicario on Sunday August 15, 2021 @04:44AM
(#28187188)
Aren't software implementations royalty free anyway? and even if they weren't, could they not classify a neural network as a software implementation? (from the patent enforcing point of view) Because if that is not the case, this idea would be applicable to a lot of things right? Seems like an easy hack