Catching up for how long though? Large models are very expensive to train, and the only reason any of them are open is because ostensibly for-profit companies are absorbing the cost on behalf of the open source scene. What's the plan B when those unprofitable companies (or divisions within companies) pull up the ladder behind them in pursuit of profits?
Nobody is advocating switching their coding agents to open source (yet), but that's not the bulk of the tokens in companies that have automated workflows integrated into their business.
Catching up enough to execute a business successfully on the open source (and free as in free beer) alternatives to OpenAI. Once you have a single model that works, you can bootstrap your own internal ML infra for just that one use case and grow from there.
[1] I asked Gemini and apparently it’s quite common
To paraphrase, Meta is not releasing open models out of the goodness of their heart, they're doing it as part of their business strategy.
- They already have all the eyeballs.
- They want to be able to run the best model they can for those eyeballs.
- It doesn't make a difference to them whether others have a similar model or not. They don't care.
- Ergo, they release their weights, hoping someone can help them improve it. Which has happened: see quantization.
Unless the actual race is to create an AI Employee that operates and can deliver work without constant supervision. At that level, of course it would be cheaper to pay $2,000 a month straight to Open AI vs hiring a junior SWE
Or is this the case of every HN discussion where what you do is “actual work” and what other people do is “toying around?”
I agree with this... for now. But the hosted commercial models aren't widening the gap as far as I can tell, if anything it appears to be narrowing.
And if the relative delta doesn't increase somehow I don't see any way in which the "AI race" doesn't end in a situation where locally run LLMs on relatively cheap hardware end up being good enough for virtually everyone.
Which in many ways is the best possible outcome, except for the likely severe economic effects when the bubble bursts on the commercial AI side.
I don't know what will happen tomorrow, let alone 1 year or more down the line.
If it ever becomes economical to run and maintain bare metal GPU compute to run LLMs, then that's what will need to be done..
When I was a SWE, I've done migrations between bare metal to AWS to GCP and then both, and then all plus Azure..
It is the cost of doing business. You pick what works best at a price that is optimized for your business needs now. You have a war chest so when they start becoming assholes, you have leverage to fight back or pivot.
I'd rather spend $200-400/mo to unblock myself NOW than do something dumb with 5 or even 100 tokens per second output that isn't actually that good as what the current providers offer. I'm going through millions of tokens a day.. I couldn't do that with local "RIGHT NOW" (<--- important)
Just because open source models are almost as good, doesn't mean you can underestimate the convenience factor.
Both can be true: we're in an AI bubble, and the large incumbents will capture most of the value/be difficult to unseat.
At one point you could get a Netflix subscription and it was convenient enough that people were pirating less. Now there's so many subscription services, we're basically back to cable packages, paying ever increasing amounts and potentially still seeing ads. I know I'm pirating a lot more again.
Uber vs cabs, Airbnb vs hotels - We've seen it time and time again, once the VC cashflow/infinite stonk growth dries up and they need to figure out how to monetize, the service becomes worse and people start looking for alternatives again.
I also wonder whether, similar to bitcoin mining, these things end up on specialist ASICS and before we know it a medium tier mobile phone is running your own local models.
(Unfortunately... only Apple Intelligence right now...)
People love to quote Dropbox ignoring all of the YC companies that are zombies or outright failed. Just looking at the ones that have gone public.
https://medium.com/@Arakunrin/the-post-ipo-performance-of-y-...
When there's real money to be made investing in YC is off limits to the public: https://jaredheyman.medium.com/on-the-176-annual-return-of-a...
They also didn’t have the massive fixed cost outlays nor did they have negative unit economics that OpenAI has.
https://news.ycombinator.com/item?id=42392302
(Before discussion of your comment devolves into nonsense about this.)
A good number of people used to pay for email. Now a tiny fraction does. It all hangs on wbether OpenAI can figure out how to get ad revenue without people moving to a free competitor without them - and there will be plenty of those.
There will be equivalent models that are free. Likely, there will even be free ones without ads.
Free+ads can beat free without ads on pure incumbency, marketing and convenience. Most people don't use an adblocker, even though it's trivially easy to install.
Paid, however, can't beat free+ads. Too much friction.
They pay for the hardware and electricity /s.
Plenty of people don't. That's an enduring advantage of using GPT over anything locally hosted.
That right there is why they are valuable. Most people are absolutely incompetent when it comes to IT. That's why no one you meet in the real world uses ad blockers. OpenAI secured their position in the mind share of the masses. All they had to do to become the next google was find a way to force ads down the throats of their users. Instead they opted for the inflated bubble and scam investors strategy. Rookie mistake.
The reality is they're a paid service, and even if they 10x their prices they're still in the red.
Consumers do actually care about price. They will easily, and quickly, move to a cheaper service. There's no lock in here.
You make another chat window and you're golden.
I said everyone knows ChatGPT you responded about OpenAI
The downside to making a zero-effort product is that consumers can easily just switch to something else with no thought. ChatGPT has literally nothing that makes it unique.
https://backlinko.com/ad-blockers-users
And just because you have users doesn’t mean it’s easy to create a profitable ad business - ask Yahoo. Besides we still don’t know how much inference costs. But there is a real marginal costs that wouldn’t be covered by ads. They definitely couldn’t make enough on ads to cover their training costs and other costs.
For more subtle and slimier way of doing things, boost the relevance of brands and keywords, and when they are semantically similar to the most likely token, insert them into the response. Companies pay per impression.
When a guardrail blocks a response, play some political ad for a law and order candidate before delivering the rest of the message. I'm completely shocked nobody has offered free gpt use via an api supported by ad revenue yet.
The hardware required to run something like deepseek / kimi / glm locally at any speed fast enough for coding is probably around $50,000. You need hundreds of gigabytes of fast VRAM to run models that can come anywhere close to openai or anthropic.
To my deep disappointment the economics are simply not there at the moment. Openrouter using only providers with zero data retention policies is probably the best option right now if you care about openness, privacy and vendor lock-in.
If I was working with sensitive data I sure would only use on prem models.
> but he completely misses the only real threat to either Open AI or Anthro.
Hard disagree. Economics matter, in fact more than tech.
Tech only gets to shine if the economics work out.
> Then again, open AI is obviously ran by some of the smartest people on planet Earth, with other extremely smart people giving them tons of money, so I can be completely wrong here
Nope.
Money != Intelligence
Open AI, SF and the West-Coast VC scene is run by very opionated, incentiviced people.
Yes, money can make things move, but all the money of the world don't matter if your unit econonmics don't work out.
And the startup graveyard is full of examples of this kind.
Open source models like deepseek and llama 3 are rapidly catching up, if I can get 90% of the functionality for significantly less ( or free if I want to use my own GPU), what value does open AI really have .
I'm a paid subscriber of open AI, but it's really just a matter of convenience. The app is really good, and I find it's really great for double checking some of my math. However I don't know how they're going to ever truly become a corporate necessity at the prices they're going to need to bill at to become profitable.
Then again, open AI is obviously ran by some of the smartest people on planet Earth, with other extremely smart people giving them tons of money, so I can be completely wrong here