Of course, there might be a wrinkle or two. How do you propose to evaluate the size of a page when large amounts of something like a newspaper article is loaded by reference, dynamic, and depends on third parties making independent run-time decisions? How can you know a page's size won't vary 50% minute-to-minute in a world like ours? And how can you meaningfully measure load time in such a context?
You're absolutely right. Page size and speed could absolutely be better ways to do this! It's just maybe possible that there could be some minor obstacles to doing so.
You downrank them immediately because that's slow.
Of course, there might be an issue here because the amount of things that work that way is huge. So now you have a scenario where everyone is angry at Google for trying to dictate how they can build web pages and writing angry digital polemics about how this is an unreasonable standard and abuse of power. Nobody actually wants to re-implement massive chunks of how their website works, so everyone will resent this incredibly artificial imposition.
Which is to say it's a wonderfully straightforward answer, but perhaps not better than AMP in practice.
But we're already doing that because Google downranks results that don't use AMP. We're generally OK with Google downranking sites on actual metrics (such as HTTPS) but not when they're pushing their "solution" that clearly has a number of issues with conflict of interest.
Personally, I prefer AMP for security reasons. It's tightly restrictive and does a lot to limit the available space to mount attacks aimed at browsers. But I understand that's far down most people's lists, and tends to fall under the same sentiment as "devs should just write fast websites".
Again, please accept my apologies for my failure to communicate my point clearly.