> Could it not tell if it's actually managing to play and keep up with time? Similarly to how (I believe) it'll degrade if the network is such that it's not loading 4k say quickly enough, it'll switch (if 'auto') to 1080p or whatever.

Probably, but it's pretty poor UX if videos start super-choppy by default and slowly downscale to a playable resolution over several seconds. Having a good default is still important.