News for hackers. Stuff that matters.
Re:Microsoft AI: Seven new world-class models at scalable pr... by rayiner on Wednesday November 18, 2026 @03:24PM (#50551648)
[delayed]
Re:Microsoft AI: Seven new world-class models at scalable pr... by andriy_koval on Wednesday November 18, 2026 @04:25PM (#50552323)
> LLM models have no such barriers.

compute and amount of training data built on top of XXX B investments is very strong barrier.

Re:Microsoft AI: Seven new world-class models at scalable pr... by criemen on Wednesday November 18, 2026 @06:23PM (#50553783)
For frontier LLMs? Sure. A almost prehistoric datapoint by now: Training costs for GPT-2 levels of "intelligence" fell by 600x over 7 years, as measured by Karpathy in Jan 2026, source https://x.com/karpathy/status/2017703360393318587

If we can extrapolate from that (I don't know if we can), good enough models might be achievable with much less than XXX B investments.

Re:Microsoft AI: Seven new world-class models at scalable pr... by andriy_koval on Wednesday November 18, 2026 @06:35PM (#50553892)
I don't think we can extrapolate evolved complexity. I think frontier labs have many thousands researchers crafting and testing various post-training tuning datasets with impact on end quality, which was not the case in gpt-2 time.
Re:Microsoft AI: Seven new world-class models at scalable pr... by VirusNewbie on Wednesday November 18, 2026 @04:59PM (#50552796)
>LLM models have no such barriers.

I think it's much more like cloud, where sure, there aren't any single barriers to buying some servers, putting K8s on it, having an account management/billing system...but in practice the expertise in aggregate to run a profitable one is quite rare.