I'm writing this from the OS's perspective. Linux abstracts this away, but we have to do it to get the same abstraction. You can see under the user space portion how simple it is to grab a framebuffer, which is then mapped into user space. Then we have full access to RGBA values.