The Rpi4 plays 720p video full screen at 5% cpu usage (because it has hardware decoding support) - using omxplayer. The hard part is for the hardware decoding support to be enabled in the right places and it seems like kodi is the only place (mpv, vlc don't do it by default)

The other missing piece for the transition to fullscreen is a robust compositor API, such that the same compositional layer can just have its size changed rather than going from app-compositor to system-composited in a new window. Something like, say, Wayland. <insert that whole debate/debacle here>

It absolutely should only take a couple of frames to make a video fullscreen, but to put things into perspective, on my 2015 macbook it takes 3-4 seconds to toggle fullscreen on youtube, and my much more powerful linux desktop isn't much faster.

Have you considered that the compression algorithms for video are nontrivial? I mean I'm sure there can be dedicated hardware for video decoding, but would you put that in a low-cost, low-energy entry-level CPU where you want to shave off as much cost in favor of flexibility?

The Pi is not primarily intended for media consumption. That said, for a similar price to a RPI + display you can get a cheap tablet that will (should?) have dedicated video hardware.

The original Pi CPU was intended for media consumption in set-top-boxes; it was a huge video core with a small ARM on the side.

It has had hardware video decompression for multiple revisions.

In fact it had H264 decoding in all revisions, optional unlockable MPEG2 and VC1(?) up to and including all Pi3 models and H265 since the Pi4.