The trends around compute increasing is on the whole correct but by no means is it a universal rule. More optimized training procedures and model architectures are coming out all the time. In just the last week, we got F5-TTS which is trained on twice as much data as the previous leader in realistic TTS (Tortoise TTS) and is exponentially faster - taking only 3 weeks on H-100. We also got Meissonic, a text-to-image model that is exponentially easier to train than any existing model. IE you can train a Stable Diffusion like model from scratch on consumer hardware or in the cloud for abou $500.

https://huggingface.co/MeissonFlow/Meissonic

https://github.com/SWivid/F5-TTS

The reason the trend is that compute costs are doubling is because this is an arms race and everyone in the corporate space is prioritizing bigger models over better architecture in the pursuit of a breakaway. It is not indicative of a law ala Moore's Law.