Shot or scene based detection can be used to place your I, P, and B frames in better locations for example.
For those that don't know, an I frame is a keyframe, a still image. P and B frame are predictions off of other frames (such as a keyframe). Frames within the same shot or scene are likely to be similar to one another, so having awareness of that contextual information can help you place for example a keyframe at precisely the start of that shot or scene, and predict everything in that shot or scene off of that.
The main advantage of ML for video compression is that people accept giant ML models, so you can hide a lot of common "features" in there. The disadvantage is that classical codecs are macroblock-based for a reason; you can't just process a whole frame at a time, nobody has the memory bandwidth for that.
I think it's not so abstract at all. The relationship feels fundamental.
Encoder, decoder, codec. Compression, entropy, lossiness, ...
There's been some cool stuff in the speech area: https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-...
Also from Microsoft: https://techcommunity.microsoft.com/t5/microsoft-teams-blog/...
There's actually a pretty nice read from one of the key figures in data compression field - Matt Mahoney: "Text Compression as a Test for Artificial Intelligence" [1]. Some interesting reads are also on his personal website, though a bit outdated now: https://mattmahoney.net/dc/rationale.html
[1] https://static.aminer.org/pdf/PDF/000/014/009/text_compressi...
There's been rumblings about how Google's AI edge would lead to jumps in video and image coding but nothing big seems to have happened yet, though lots of micro-optimisations appear to be ML based.
Some suggested that the AV1 throw a bunch of tools at it spec design was built for AI to use.
Possibly VMAF is the biggest ML impact on codecs so far.