Gosh, I'd love to find my old Computer Architecture Professor's slides on the like 3 axes on which you can improve Computer instructions or mflops. But in general my recollection is you can do the following (please except 1 salt rock I'm going from human memory):

1. Transistor size (This is the official Moore's law) that we're improving the number of transistors per area.

2. Clockspeed (Pretty sure this just scales with power, and I'm like 99.9% certain that they're just making gas guzzlers over at nvidia that show improvements in the top-speed, because they use more power).

3. Instruction Throughput - Improving your ISA to be more efficient is something that we've generally been getting better at. It's also why microarchitectures exist, as I understand it (though someone can correct me on this).

I feel like NVIDIA is just saying "We can ramp up #2 all day, no one has really explored how much power one of these things can take" when in fact power-draw is potentially the worst way to try and improve this. (Or at least, they will likely reach a similar limit by dialing up this piece). How far turned up companies are on these dials is kind of how I would expect to evaluate how much growth a company has before they start to really feel the limits of CPU/GPU performance. But also, I would consider it concerning if companies weren't actively investing in all 3 of these, because I think it would be much more likely for them to hit the limits of 2 of these curves and stall out while they're trying to figure out how to improve performance on the other 2.