Why not used shared memory? I believe that's what Chrome uses for its rendering buffers. The renderer is sandboxed and shares buffers with the X11 process.
EDIT, used to use: https://codereview.chromium.org/298443002/
Still, it appears it used this method for a long time, so it seems feasible.
Also, Chrome has multiple video playing paths, including Flash, which is sandboxed, so I don't see the difference. The Flash video has to be decoded in a sandbox. I imagine HTML5 video is sandboxed somehow too.
It is very platform-specific and difficult, and performance is a prime concern, but not impossible.
Because this is not normal shared memory, this is GPU shared memory. This is hard to port.
I was afraid that this was the case. Might shared memory be a viable solution to buffer sharing?
In any case, it makes me really happy to know that you've thought about this!
It's GPU allocated memory, so not so sure.
> In any case, it makes me really happy to know that you've thought about this!
If you have ideas, we'd love to work on it, seriously.
Did you look into the new memfd support in the kernels? Seems like this is a pretty cool way to do process splitting while sharing memory in a "safe" way. (i.e. the recieving side can guarantee that the sending side is not playing silly-buggers with the buffers while you read them).
For hardware decoders, maybe using dmabuff buffers shared between multiple processes would work?
This is not portable, but this is a possible solution for Linux.
But this is far from being ready, which is one of my main point :)
> Does VLC currently have pretty clear separation between its various components?
Very clear separation. One of the best, tbh.
> Do you think it be much work to spin security-conscious parts like the decoders into separate processes?
Extremely difficult. We've thought about it.
For a video player the 3 parts that are sensitive, are protocols (file, http), demuxers (mkv, avi) and decoders.
The crashes mostly happen in protocols and demuxers, but not in decoders (a contrario from what people think).
The main issue is that the video decoder MUST be in the same process than the video output, for performance reasons (buffer sharing: memcpy is murder) and for hardware decoders. And video output are usually with very high access in the kernels. Moreover video outputs are almost necessarily in the process with the UI thread.
For audio output, it's not good either, although some platforms are better (pulseaudio+Kdbus might work).