The gravitas will shift to enabling matrix math rather than algorithms being designed for special hardware.
Matrices are the new paradigm.
My point was that if training could emit an efficient video codec as a network of logic gates, rather than as a dense array of neural network weights, it might just produce something practical for video playback on mobile devices and video processing at hyperscale.
That is exactly the same path that led us to GPGPU, which in turn derived to using GPUs to accelerate neural nets. In every step of the way it was about repurposing existing hardware that was designed for something else and was thus suboptimal.
A small number of videos are going to be viewed millions of times. For these few videos it’s worth the compute to do the best possible compression.
There’s no reason the same video formats and decoding hardware can’t be used for both. The high compute version can use an ML model to figure out how to optimize the bandwidth budget across every scene while the cheap version uses some easy heuristics.
This is a common incorrect assumption. You have to balance ratio with speed, otherwise you end up with something that spends more time decompressing than actually downloading, as is literally the case today with XZ. this is why alternatives like Zstandard are getting popular. 9 times faster decompressing for 10% worse ratio.
Humans not only design for compression quality, we also really focus on cost. And we design direct to what is cheap in hardware: integer adds, shifts, binary logic.
I do believe a different paradigm of ML is necessary for this particular application. The neural primitives used need to be more closely related to transistors than to matrix multiplications.