Local models that run on a laptop (not even needing a "cluster") are already better than ChatGPT from a couple of years ago. Yes, Claude and ChatGPT today are certainly better than these local models, but they can't keep getting better indefinitely -- there's only so much info to scrape. When they hit a plateau, it is only a matter of time that consumer hardware will catch up to it.
> but they can't keep getting better indefinitely
Maybe? We dont really know this right? People have been saying this for 5 years now and the models are still getting better. The companies running the frontier models have already scraped everything on the web, but the models are still getting better, even if it's only marginally better, with each release. Maybe eventually some company will actually achieve AGI/ASI, who knows..
While that's most likely true, it rests on the assumption that consumer hardware stays affordable enough, and isn't locked down to disallow running "untrusted" models. I would have never believed that these assumptions could ever turn out false, but the recent developments have shown that even if unlikely, it's not impossible.
I think the parent is speculating that there may be an order of magnitude improvement in the cheap / OSS model space such that one running on a piddling desktop cluster could match or exceed the capabilities of the current SOTA on billion-dollar datacenter.
> I think the parent is speculating that there may be an order of magnitude improvement in the cheap / OSS model space such that one running on a piddling desktop cluster could match or exceed the capabilities of the current SOTA on billion-dollar datacenter.
And then they take that model, put it in a billion-dollar datacenter, and kick your desktop cluster's ass with it.
What makes you think your "cheap or open source model" running on your piddling desktop cluster will be able to complete against a SOTA one running in a billion-dollar datacenter?
It's a cyberpunk fantasy. It won't work out that way.