AI methods are improving all the time. People think that modern chess engines are more powerful due to hardware advances but search refinements are enough - SF on a Pentium II would destroy even Kasparov and Deep Blue. In a like manner the guys at the frontier labs discovered an indirect method for attacking many problems and we can expect it to receive considerable refinement. It will not permanently require expensive training.

There was a rather interesting article (sadly LLM-written and not very well either, because the underlying content would be actually good) about this era being "AI's 2 FPS moment".

The comparison was made with the graphics engine team at Microsoft who were writing the first version of DirectX at the time when then-current hardware could just about do 2 frames per second. The sentiment apparently was along the line "why would you bother?" By the time the release grade stack shipped, hardware had caught up and 30 FPS was commonly doable.

We should expect to see similar step-change improvements over time, as the algorithms are refined and hardware further adapted to the new bleeding-edge constraints. In a way we already are, because apart from the frontier models ballooning like crazy, the "regular" class models are becoming more capable in smaller sizes.

Training a human takes much less money and data than training an LLM. Even if I spent my entire life doing it, I would not be able to read the entire corpus of any LLM, even counting just the English subset. LLMs need so much more because they start out as blank slates, while we don't.

yes, but LLMs can hold all this data and once they are trained they blow any human out of the water. It takes a few years for an LLM to ingest the entire internet. No human will ever be able to do that.