Funny, my smallish ARM Cortex-M / VGA DIY video console also works like that, with less RAM than needed for a framebuffer. There are framebuffer modes, but with less pixels/bpp than natively output by the video signal output.