If we had standard headers, then reading metadata wouldn't be part of the decoder. The decoder would only need to take in bytes and output a bitmap, or take in bytes and output PCM audio. It doesn't need to be able to call any functions, or run any system calls, and the data it outputs can safely contain any bytes because nothing will interpret it.
It's like taking the very core of webassembly and then not attaching it to anything. The attack surface is astoundingly small.
You just need to give it a big array of memory and let it run arithmetic within that array, plus some control flow instructions. Easy to interpret, easy to safely JIT compile.
The part of sandboxing that's hard is dealing with I/O, or giving useful tools to the sandboxed code, or implementing data structures for the sandboxed code. You don't need any of that for a multimedia decoder. You just let it manipulate its big block of bytes, and make sure you bounds check.
A Java VM exposes tens of thousands of functions to the code inside it. A barebones sandbox exposes zero. It just waits for the HLT opcode.
And when it gives you raw RGB data, or raw PCM data, there's no way to hide a triggerable malicious payload inside. If the code does something bad, the worst it can do is show you the wrong image.
But my suggestion would be that you build a video codec out of it. Preferably one that has the properties the market demands: performance and energy efficiency.
You'd use existing codecs, and the way you get good performance and energy efficiency on a video codec is by having a hardware implementation. Software decoding doesn't even come into that picture.
As far as practical software decoding outside of battery-powered video, can I just point at webassembly? Especially the upcoming version with vector instructions. You could use normal webassembly, or even an extra-restricted version. It gets pretty good performance, and when you remove its ability to talk to the outside world it goes from pretty good to extremely good security.
WebAssembly codecs indeed exist, and they are impractical due to a lack in performance.
The container parser would not be dynamically downloaded, and may or may not be sandboxed.
We don't need a new container with almost every codec. We just need the new codec itself.
> WebAssembly codecs indeed exist, and they are impractical due to a lack in performance.
Mostly because they don't have vector instructions yet, I bet. But plenty of webassembly is within 50% of native, which is good enough for lots of things, which includes image decoding for sure.
The challenge remains for you to actually provide the codec you describe. Which a few comments ago was trivial because it was a hardware codec anyway, now it’s just a bit of WebAssembly away. Well that should be trivial because cross compilers to WebAssembly exist. So why don’t you just provide a few real world examples? Your probably not the first to think of these ideas, there has to be a reason why it hasn’t been done yet.
Not "magically". But you only need one or two, and they don't need to be very fast, so you can put a lot of effort into making them secure.
But more importantly, browsers already have many container decoders. This is not an expansion in attack surface. The goal here is allowing a lot more codecs compared to current browsers without a significant increase in attack surface compared to current browsers. Pointing out flaws that already exist doesn't disqualify the idea.
> So why don’t you just provide a few real world examples? Your probably not the first to think of these ideas, there has to be a reason why it hasn’t been done yet.
Image decoders in webassembly already exist. Did you even look? Including JXL!
Video decoding needs more support structure in the browser, and I already said some decoders need things that are being added to webassembly but aren't done yet. Even then, the first google result for "av1 webassembly" is a working decoder from five years ago.