Re:Microsoft AI: Seven new world-class models at scalable pr...
by criemen on Wednesday November 18, 2026 @06:23PM
(#50553783)
For frontier LLMs? Sure. A almost prehistoric datapoint by now: Training costs for GPT-2 levels of "intelligence" fell by 600x over 7 years, as measured by Karpathy in Jan 2026, source https://x.com/karpathy/status/2017703360393318587
If we can extrapolate from that (I don't know if we can), good enough models might be achievable with much less than XXX B investments.
Re:Microsoft AI: Seven new world-class models at scalable pr...
by andriy_koval on Wednesday November 18, 2026 @06:35PM
(#50553892)
I don't think we can extrapolate evolved complexity. I think frontier labs have many thousands researchers crafting and testing various post-training tuning datasets with impact on end quality, which was not the case in gpt-2 time.
compute and amount of training data built on top of XXX B investments is very strong barrier.