IIRC Flipboard once tried this approach, rendering everything to a <canvas>. As it stands right now, that will never be a viable solution for the reasons you mentioned, along with the fact you throw literally all accessibility features out the window.