This forces ant/oai to lower their prices but how much that is subsidized nobody knows. It might very well just be they are in deep trouble, they see their clients leaving, they have to lower their prices.
But let's not forget openai is riding on the edge of bankrupcy, their last investment round apparently gave them few months of runway, they are leaking money. I don't think they know how to turn this around with all the open models being competitive.
I would just enjoy cheaper model costs while they are like this.
I feel open-weight was able to keep up the past year using efficiency improvements (hybrid attention, improved MTP drafters, improved optimizers) to offset having less compute. But OpenAI/Anthropic started cutting token costs just a few months after these, which to me suggests they copied these improvements. Indeed it would be hard to work in LLM training and not know about DSpark, Muon etc.
So what do you do if closed source uses your learnings, gives no knowledge back, and has 10x more compute and funding on top of it? The situation feels quite dim right now, esp. with the Nvidia thing.
Chinese AI also has a clear business mode: making money from its (i.e., chinese AI) applications.
I'll argue that the latter is closer to the end-goal here than the former.
If we extrapolate over the next several years, prices for VRAM/RAM will start to come down as manufacturing capacity comes online, models will continue to become more token efficient, and intelligence will improve at every quant size. As this happens, more and more people will be running LLMs locally, and there's going to be a point where local can do everything you need to do unless you're literally trying to do breakthrough mathematics research.
> and there's going to be a point where local models can do everything you need to do unless you're literally trying to do breakthrough mathematics research
Maybe. But competitive markets pay exponentially more for incrementally superior services. Professional athletes are a great example of this. If you're building a product with competitors and those competitors have frontier models and you stick to local then you're probably going to lose both today and in the near-ish future.
If you're bitter lesson pilled then the alpha is entirely in having a better model, more data, and more compute. Your vision is that human ingenuity will provide more alpha. Oof. I wish. But my tastebuds are very very bitter.
Just this week, qwen 3.8-27b got two new interesting quantizations, one of which came from community. Llama.cpp also improves continuously, and novel alternatives like Strata pop up.
i highly doubt this premise. the success of a product has very little to do with the amount of the intelligence used in producing it. you could throw millions of dollars using AI to develop a movie and it would still flop in comparison to an indie-made film.
I want to use "more intelligence" but not in terms of single higher IQ.
I want to use more intelligence in terms of a sum quantity of knowledge work being performed. A team of 10 instead of a team of 5.
This is when a local model will shine. A fixed cost and I don't have to worry about how much I am using it. I already can't tell any difference at all between Opus and Fable. It won't be that long that I won't be able to tell the difference between a local model and Opus.
There will be an infinite demand for "more" intelligence but I think many people are making a category error on the dimension in question.
the lesson is that training a model on all input ever output will never be as good as training a model on arbitrary outputtable input.
Once I have a model locally, I know it isn't going to change on me. Compare that to using an API from Anthropic or OpenAI (and yes, the only thing you can say is you're using an API, not one model), they can discontinue a model you're relying on (as every model could have peculiarities you've designed around) or modulate their speed/quality day-to-day to save on compute costs.
And trying to claim that buying your own hardware doesn't pay off compared to API or subscription costs doesn't hold much water either, at least not for me. For starters, I already had the GPUs and machines I started running models with, whether it was for gaming or for other reasons. And in the case of the new GPUs I bought recently (2x AMD Radeon Pro R9700; no you do not need CUDA and you haven't for a long time now), I can hold a reasonable expectation that they will hold at least some of their value years from now, compared to paying for API/subscription usage for years in which case you're left with nothing.
So, if you have a local model that is "good enough" (and they already exist for many use cases), it's just kind of the obvious choice to make, IMO. And in just one person doing it as a hobby while I'm still looking for a job 16+ months after a layoff. A small business and certainly a large enterprise has even less reason to open themselves up to all the downsides that come with using Anthropic or OpenAI.
why more like hardware? takes very long time to develop, expensive to manufacture (run), valuable.
.. if exponential, they’ll never catch up but if logarithmic, closed models will plateau and so catching up is inevitable.
My money is on a plateau
I hope it will plateau
Following that logic the gap widens a little bit here but then will steady again. Before this there has been a general catch-up trend too which probably also persists leaving us in a slow closing of the gap as the net effect?
This doesn't take other factors into account. Compute capability, infrastructure capability (the chips can't work without electricity!) etc.
Those factors translate into a ceiling for this hypothetical exponential speedup.
So, with new models coming out on a literally weekly basis, I think trying to figure out a future trajectory is just going to be a wild guess. For all we know, Anthropic and OpenAI models may also be plateauing in the near future; or there may be a breakthrough and they may improve beyond our wildest beliefs. I honestly don't know.
That said, I suspect available compute and model sizes that major Western vs. Chinese labs can handle will still keep the gap open at least in the short term.
On the other hand, I am also not sure how much the gap will matter past a certain point. I am specifically wondering at what point people will become interested in smaller models fine-tuned for specific application areas (e.g. cheminformatics) rather than massive generalist models.
Their open weights models are the the smaller ones, which aren't released yet.
Lots of research goes into open-weights, optimizing cache re-use, storage, inference costs, etc., and all these benefits are easily rolled into closed AI models. Meanwhile any advances made by OpenAI, Anthropic, etc. are kept hidden away to give their models an edge.
It'd already be a miracle if open-weights were gaining on-par with closed-weight models. That they've been closing the gap for most of the past year is wild!