News for hackers. Stuff that matters.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by evo_9 on Sunday May 17, 2026 @10:31AM (#48169256)
Every AI subscription is a ticking time bomb for the frontier provider; within a few years we will be running local models as good as today’s frontier models with almost no cost burden. The floor will fall out of the enterprise market for all the frontier companies.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by crazygringo on Sunday May 17, 2026 @11:20AM (#48169716)
> within a few years we will be running local models as good as today’s frontier models with almost no cost burden

Based on what? The RAM requirements alone are extraordinary.

No, running large models on shared, dedicated hosted hardware at full utilization is going to be vastly more cost-efficient for the foreseeable future.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by alsetmusic on Sunday May 17, 2026 @11:26AM (#48169778)
Local modals are 6 months to 18 months behind frontier. Even if the performance of a cloud model is faster, it's clear that local is catching up.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by alecco on Sunday May 17, 2026 @12:12PM (#48170214)
> Local modals are 6 months to 18 months behind frontier.

I wish this was true but it is not. And I am working on open source models so if anything, I would have a bias towards agreeing with you.

Frontier closed models (GPT/Claude) are gaining distance to everybody else. Even Google, once the king.

Your claim is a meme coming from benchmark results and sadly a lot of models are benchmaxxed. Llama 4, and most notably the Grok 3 drama with a lot of layoffs. And Chinese big tech... well they have some cultural issues.

"Qwen's base models live in a very exam-heavy basin - distinct from other base models like llama/gemma. Shown below are the embeddings from randomly sampled rollouts from ambiguous initial words like "The" and "A":"

https://xcancel.com/N8Programs/status/2044408755790508113

---

But thank god at least we have DeepSeek. They keep releasing good models in spite of being so seriously resource constrained. Punching well above their weight. But they are not just 6 months behind, either.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by dools on Sunday May 17, 2026 @12:28PM (#48170398)
Kimi k2.6 is about on par with GPT 5.2 so I’d say open weight models are about 6 months behind.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by cbg0 on Sunday May 17, 2026 @12:57PM (#48170679)
The Q4 quantization requires about 600GB of RAM without context, not exactly consumer hardware friendly.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by janderland on Sunday May 17, 2026 @12:54PM (#48170653)
Has Kimi found a way to vastly reduce the amount of VRAM required without running at 3 tokens per second? That’s the real concern.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by dools on Sunday May 17, 2026 @07:56PM (#48174225)
I said "open weight" rather than "local". I mean, local if you have $240k to drop on GPUs but you can run Kimi k2.6 on a B300 cluster for ~$50/hour too.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by tyre on Sunday May 17, 2026 @12:35PM (#48170481)
The Chinese models should stay close on a lag. They’re doing a ton of distillation that, realistically, I’m not sure the American frontiers can stop.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by alecco on Sunday May 17, 2026 @01:38PM (#48171106)
US labs got tough on "adversarial" distillation [1]. I suspect that's one of several reasons why Chinese big labs are lagging again.

[0] US AI firms team up in bid to counter Chinese 'distillation' (Apr 7) https://finance.yahoo.com/sectors/technology/articles/us-ai-...

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by tyre on Sunday May 17, 2026 @04:22PM (#48172867)
Yeah I mean the US has gotten tough on, like, foreign interference in elections and cyber security, but if you have the Chinese state behind you—which they absolutely do and as an observer, obviously, they have to—no company can stop them.

Case in point: North Korea, with far, far fewer resources.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by crystal_revenge on Sunday May 17, 2026 @01:58PM (#48171373)
I’ve worked, for a long time professionally, in the open model space for 3 years and up to 2 months ago I would have agreed with you. But it’s empirically not the case today. These models (combined with a good harness) have dramatically improved in both power and performance.

Gemma 4 was a major improvement is self-hostable local models and Qwen-3.6-A34B is a beast, and runs great on an MBP (and insanely well on a 4090).

The biggest lift is combining these models with a good agent harness (personally prefer Hermes agent). But I’ve found in practice they’re really not benchmaxxing. I’ve had these agents successfully hand a few non-trivial research projects that I wouldn’t have been able to accomplish as successfully even last year.

When you add in the open-but-not local models, Kimi, GLM, Minimax, you have a lot of very nice options. For personal use anything I don’t use local models for I give to my Kimi 2.6 powered agent.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by alecco on Sunday May 17, 2026 @04:57PM (#48173131)
For specific use cases, absolutely, a harness and other techniques help (this is literally what I'm working on). But GP was talking about general use.

Over-promising is a very stupid thing. Nobody will value the intermediate steps. Nobody will value all the effort because they will always compare us with frontier models made with billions and we will become a running joke. So please stop.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by alsetmusic on Monday May 18, 2026 @08:20AM (#48178694)
Over-promising is what the frontier companies are doing. I'm not pretending open weight models are gonna do your homework and pay your taxes and remember your wife's bday with a super personalized gift. I'm just saying that they seem pretty good for what they are. There's no promise being made here.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by datadrivenangel on Sunday May 17, 2026 @04:54PM (#48173108)
Local models are ~18-24 months behind the frontier on approximate intelligence, and then like 36-48 months behind the frontier on inference speed for nice hardware.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by greesil on Sunday May 17, 2026 @11:29AM (#48169805)
How do you know this? I'm not trying to attack your statement, I am genuinely curious how anyone knows anything about model performance outside of benchmarks that are already in the training set.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by scragz on Sunday May 17, 2026 @11:52AM (#48170011)
using them you kind of get a feeling for skill level and can extrapolate that better than juiced benchmarks.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by __s on Sunday May 17, 2026 @11:56AM (#48170044)
You still need the hardware

I've got a strix halo staying warm at home, it has nothing on top models with big budget. It's good supplement to low end plans for offloading grunt work / initial triage

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by manmal on Sunday May 17, 2026 @12:00PM (#48170091)
Have you looked into DwarfStar 4?
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by __s on Sunday May 17, 2026 @12:15PM (#48170252)
Been away from home for nearly a month, so was mostly going off Qwen 3.5 122b-a10b (Q4?) / Qwen 3.6 35b-a3b (Q8) / Gemma4 31b (Q8)

Thanks for suggestion tho, tool by antirez is always going to pique interest, I'll check it out when I'm finally home again

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by manmal on Sunday May 17, 2026 @01:56PM (#48171340)
His quant that fits into 128GB looks interesting for Spark DGX as well IMO.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by lukeschlather on Sunday May 17, 2026 @11:53AM (#48170024)
It is not getting easier to obtain hardware that can run models which are sufficiently useful to undercut frontier models, if anything the cost of such hardware has gone up by 25% or more just in the past 6 months.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by aleqs on Sunday May 17, 2026 @12:58PM (#48170689)
I think hardware prices will come back down once we start seeing more efficiency improvements in models and hardware, and once more people and companies self-host models (which seems to be happening more and more these days). I think the massive infra/hardware expenditures of OpenAI and the like are going to end up unnecessary, leading to hardware price drops.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by t-sauer on Sunday May 17, 2026 @01:09PM (#48170801)
If companies decide to self-host, wouldn't that drive the demand and therefore prices up? Most companies currently do not have the needed infrastructure.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by aleqs on Sunday May 17, 2026 @02:08PM (#48171504)
I think companies will self host (including on rented hardware) even if it's more expensive, and that, along with efficiency improvements, will drop demand for big AI. I think big AI is overspending on hardware/datacenters at the moment.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by calvinmorrison on Sunday May 17, 2026 @11:29AM (#48169808)
if that's true - and in 6 or 12 months i can get what i have today, it might not be worth paying anthropic.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by toasty228 on Sunday May 17, 2026 @02:40PM (#48171880)
> Local modals are 6 months to 18 months behind frontier.

At what tps? You can run the new gemini flash or 5.3 codex spark at 1000+tps and run circles "open" models. You can't run anything useable locally without at the very least a blackwell 6000 if not two

Sure you can run qwen 3.6 at 20tps on a mac 128gb but let's not pretend this will get you anywhere

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by leptons on Sunday May 17, 2026 @12:07PM (#48170162)
>running large models on shared, dedicated hosted hardware at full utilization is going to be vastly more cost-efficient for the foreseeable future.

That is only true right now because hundreds of billions of dollars are being burned by these AI companies to try to win market share. If you paid what it actually cost, your comment would likely be very different.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by jazzyjackson on Sunday May 17, 2026 @12:13PM (#48170228)
No, it's economies of scale and I don't understand where anyone is coming from that thinks they'll be better off buying their own hardware, why would you get a better deal on MATMULs/watt than the cloud providers ?
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by salawat on Sunday May 17, 2026 @12:41PM (#48170547)
Another victim of Goldratt's Theory of Constraints. Some things are more important to optimize for than MATMULs per Watt. What that is I leave as an exercise to the student. May you realize what it is before it is too late.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by jazzyjackson on Sunday May 17, 2026 @01:08PM (#48170792)
Some individuals will choose some $10,000 hardware so they can keep freedom and privacy and that's well and good, my point is just that freedom and privacy is not what wins marketshare, and hence, IMHO, local LLMs are not going to catch up and surpass frontier models like some in this thread like to claim
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by esseph on Sunday May 17, 2026 @01:20PM (#48170919)
> freedom and privacy is not what wins marketshare

Digital sovereignty laws may mandate/remove access to LLMs of other countries on economic and national security grounds.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by esseph on Sunday May 17, 2026 @01:19PM (#48170911)
Within 5-10 years you're going to see a box like one of those AMD Halo nodes running homes.

They'll be controlling lights and temperature, they'll be adding calendar reminders that show up on your phone and your fridge. Your phone and devices might sync pictures and videos there instead of the large cloud providers. They'll also be a media server, able to stream and multiplex whatever content you want through the home. They'll also be a VPN endpoint, likely your home router, maybe also a wifi access point.

I think this makes quite a bit of sense. I don't think they'll be ubiquitous, but they could be.

This distributes the power demand where local solar generation can supplement , gives the home user a lot of control, and claims overship of the user data from big tech.

Maybe I'm imagining things but this is what I think is coming.

It's the lmm/data heart of the home. A useful digital tool.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by marmarama on Sunday May 17, 2026 @04:15PM (#48172817)
Umm, you can do basically all of this, today, with Home Assistant and a handful of add-on apps.

I use a local LLM with it, but you can use a hosted LLM if you like.

The core home automation stuff can run on a potato. The LLM just writes new automations when I ask it, or acts as a natural language interface.

I use a pretty small 4B parameter local LLM, on a fairly modest mini PC. It doesn't take a frontier model to do that kind of work.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by esseph on Sunday May 17, 2026 @05:16PM (#48173272)
I am 1000% aware of this, but I think we're going to see more packaged solutions in the hardware front.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by saltcured on Sunday May 17, 2026 @02:53PM (#48172055)
It's amazing to me. You say this like it isn't an absolute horror. We've really ramped up the malignant bloat of the software industry if it goes this way.

We'll have this massive machine to do "home automation", something that by all rights should be possible with less computing than is deployed in smartwatches today. Yuck...

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by esseph on Sunday May 17, 2026 @03:56PM (#48172667)
Moving the LLM from SaaS to the home, reducing the power distribution problem, and giving people control back over their data - getting it away from Big Tech. The home controls should also be more responsive that most current modern home automation that mostly uses wireless and Bluetooth to a cloud service. These are all good things.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by scheme271 on Sunday May 17, 2026 @12:45PM (#48170576)
We don't know the parameters but it probably takes at least a H100 and possibly several to run a SOTA model. Given the pricing (25+k per H100 + hardware to run it) and power (700W per H100 + hardware to run it), I don't see how anyone except for a largish company can afford to run this.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by sshumaker on Sunday May 17, 2026 @01:12PM (#48170828)
Are you serious? It’s multiple nodes to run a frontier model (a node is 8x GPUs), and they aren’t running on H100s. You are looking at 32+ GPUs.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by scheme271 on Sunday May 24, 2026 @10:20AM (#48257485)
I was being pretty generous to the comment I was replying to. Needing 32+ H100s just strengthens my argument that people aren't going to run frontier models locally anytime soon.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by iwontberude on Sunday May 17, 2026 @12:18PM (#48170292)
I strongly disagree. Humans are so insanely well incentivized here with trillions in market share to make localized AI good enough and that’s the only benchmark they need.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by SkiFire13 on Sunday May 17, 2026 @01:14PM (#48170848)
Are they? I don't believe there's that big of a market for local AI. Most people don't care that much, and you'll most likely lose the advertising revenue.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by GenerWork on Sunday May 17, 2026 @02:11PM (#48171545)
>I don't believe there's that big of a market for local AI. Most people don't care that much,

I agree that the market for local AI is basically limited to nerds at this point, but that's because nobody's really explained why local AI is a good thing and also because the vast majority of people need the $20 paid plan at most. How much time and money would it take to get something half as good as OpenAIs products running locally?

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by SkiFire13 on Sunday May 17, 2026 @04:00PM (#48172692)
> that's because nobody's really explained why local AI is a good thing

There are a lot of good things that need to be explained to people, but nobody ever managed to. I don't think this will be any different.

> because the vast majority of people need the $20 paid plan at most

Exactly, people are not gonna invest time and money when there's already something else that satisfies their need.

Local AI will need to be both better and more convenient in order to be adoped by the masses.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by throaway198234 on Sunday May 17, 2026 @08:02PM (#48174263)
Free is always better.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by SkiFire13 on Monday May 18, 2026 @04:35AM (#48176866)
Linux is free too, but it hasn't replaced Windows despite lot of people explaining why it should be better.

Moreover local AI is not free. You'll need proper hardware to run it which you have to pay extra for it.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by throaway198234 on Monday May 18, 2026 @12:16PM (#48181775)
but Free Windows would outsell priced Windows if it were offered open source, a la LLMs
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by mycall on Sunday May 17, 2026 @02:30PM (#48171774)
It will take another [human] generation before AI is well integrated into everyone's daily lives where people will expect a local model handling things for them. I don't think the killer app has arrived yet (OC is a hint of what is to come).
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by datadrivenangel on Sunday May 17, 2026 @11:22PM (#48175216)
An M5 macbook pro running LM studio with Gemam4 or Qwen3.6 is basically the original GPT 3/ gpt3.5 chatgpt experience. So about $3000 usd
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by bsder on Sunday May 17, 2026 @08:50PM (#48174539)
I agree that the vast majority of punters don't care about "local AI".

However, if you can deliver 90% of the value of AI for 90% less cost, that is a really big incentive. Companies will spring up to fill that kind of gap.

Nobody can undercut the big AI players right now because they are all over-funded by VC money. Once the frontier companies try to match cost to expense, suddenly they become very, very vulnerable.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by harrall on Sunday May 17, 2026 @12:27PM (#48170386)
You can now buy 128 GB unified memory computers from AMD as commodity.

They’re still pricey, the world is still scaling up memory production, and a lot of code isn’t yet built for AMD, but we went from the Wright’s brothers first airplane to jet engines in 27 years.

I’m not sure “it’s only a few years away” but we are sure moving there fast.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by nine_k on Sunday May 17, 2026 @12:38PM (#48170519)
> first airplane to jet engines in 27 years.

Nitpick: more like 36 years, from Wright Flyer in 1903 to Heinkel 178 in 1939. Still quite impressive.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by dghlsakjg on Sunday May 17, 2026 @02:42PM (#48171908)
nittier pick: They said engine, not airplane: the first jet engine ran in 1907 (pulse jet). The first Turbojet engine ran in 1937.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by Traubenfuchs on Sunday May 17, 2026 @12:44PM (#48170573)
I believe the same thing but keep repeating the question: Then what are all the datacenters for?
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by moregrist on Sunday May 17, 2026 @01:01PM (#48170725)
Non-cynically: the frontier providers have a projection for demand.

Cynically: it’s become an executive-level gpu measuring contest. If you’re not making huge commitments on data centers, you can’t be a serious player.

Realistically: It’s a mix of the two. The recent Claude caps for agentic usage suggest that demand exceeded their immediate compute supply. That they can alleviate it with additional capacity from the existing and small-ish xAI facility suggests that either demand may not be rising quite as fast as anticipated, that they’re okay in the short term until more capacity comes online, or a mix of both.

Open questions:

1. At what price point does demand fall, and are the frontier providers overall profitable before that price point?

2. At what price/performance point do on-prem local models make more sense than cloud models?

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by harrall on Sunday May 17, 2026 @12:54PM (#48170648)
I print documents and photos at home regularly but I still contract out to dedicated print shops.

The print shop can’t replicate the practicality of local printing and I can’t replicate their scale of investment. Both coexist perfectly.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by nnoremap on Sunday May 17, 2026 @01:02PM (#48170735)
Print-outs are a physical good. Tokens aren't.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by bluGill on Sunday May 17, 2026 @01:49PM (#48171249)
They are both fungible. You can replace one with the other.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by nnoremap on Friday May 22, 2026 @07:12AM (#48234291)
How does that relate to my comment. I didn't say anything about the fungibility of either. Physical goods have wildly different logistical constraints compared to anything digital. This, and only this, I would argue, makes their production at home attractive to consumers. Tokens just don't have these properties.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by chris_money202 on Sunday May 17, 2026 @12:53PM (#48170644)
Agents
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by nine_k on Sunday May 17, 2026 @01:02PM (#48170729)
> shared, dedicated hosted hardware at full utilization

I must say that the largest dedicated hosted hardware providers now, like Amazon or Google, to a large extent do not produce the software they are offering as a hosted solution (like Linux, Postgres, Redis, Python, Node, etc). Similarly I'm not sure if the producers of the frontier models are going to keep their lead as the service providers for the most widely used models. They would need to have quite a bit of an edge above open-weights models.

Also, models are given very sensitive data to process. For large organizations, the shared dedicated hardware may look like a few (dozens of) racks in a datacenter, rented by a particular company and not shared with any other tenants.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by crystal_revenge on Sunday May 17, 2026 @01:48PM (#48171226)
> Based on what?

I take it you haven’t actually run any of the current gen local models?

They all fit on fairly accessibility hardware, and their performance is at least on par with what I was paying for last year.

I have one of my agents running entirely from a local model running on a MBP and it has repeatedly shown it’s capable of non-trivial tasks.

Playing around with another, uncensored, local model on my 4090 desktop has me finally thinking about canceling my personal Anthropic subscription. Fully private, uncensored chat is a game changer.

For work it’s still all private models but largely because, at this stage, it’s worth paying a premium just to be sure you’re using the best and it saves the time of managing out own physical servers. But if we got news tomorrow that Anthropic and OpenAI were shutting down, a reasonable setup could be figured out pretty quickly.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by Leynos on Sunday May 17, 2026 @01:56PM (#48171348)
What kind of useful context window are you getting on a 4090, out of curiosity?
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by crystal_revenge on Sunday May 17, 2026 @02:14PM (#48171581)
256k tokens for both the MBP and the 4090
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by thiisguy on Sunday May 17, 2026 @08:55PM (#48174575)
Can you share more about your setup please? Like which models and other specs on the machines?

Edit: ah I see the models mentioned in another comment of yours

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by fakedang on Sunday May 17, 2026 @06:01PM (#48173572)
Which models are you currently using?
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by crystal_revenge on Sunday May 17, 2026 @06:50PM (#48173869)
Was using Gemma-4-A3b-26B for a while for chat (using llama.cpp for backend and Open Web UI for client features). I’ve been using Qwen-3.6-A3B for agents and am currently playing with one of HauHuaCS’s uncensored Qwen models for chat and really liking it.

I also have an agent using Kimi 2.6 as a backend (which is open, but not local) and for some coding tasks as well.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by simooooo on Sunday May 17, 2026 @02:06PM (#48171489)
Qwen 3.6 is virtually indistinguishable from Claude on my 5090
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by janderland on Wednesday May 20, 2026 @10:33AM (#48208543)
What kind of codebase do you work on (number of lines?). How many tokens does your local context support?

Maybe your statement is true for smaller codebases and shorter conversations, but I’d be surprised if you actually achieve good results on millions of lines of code with a million token context.

Granted if your setup works well for your workload then that’s all you need.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by dandellion on Sunday May 17, 2026 @02:16PM (#48171607)
> The RAM requirements alone are extraordinary.

At the same time, $100 a month is A LOT of RAM.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by claysmithr on Sunday May 17, 2026 @03:12PM (#48172239)
Not really, I can run models on my 24GB mac.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by SonnyTark on Sunday May 17, 2026 @08:37PM (#48174476)
I run Qwen3.6-35B-A3B on my 8GB VRAM GPU for 3 weeks now and its been blowing my mind how good it is (coded multiple tools that I use daily, setup CI/build scripts for several projects, meaningfully contributed to a large personal project, etc).

No one can deny that right now these new compact models are not as good as frontier models but for the first time we actually have competent local-first models. If I give you a local model that runs on your current hardware and performs at 75% of the ability of a frontier private paid model, would you still pay for frontier? More importantly, would you hand control of your processes and code to them knowing enshitifcation and price-hikes are always lurking nearby?

For businesses, I get it you want to compete. But personally, it's over. Even if I considered for a second paying OpenAI/Claude, not gonna happen now.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by slashdave on Sunday May 17, 2026 @11:49AM (#48169984)
> within a few years we will be running local models as good as today’s frontier models

I seriously doubt it. Scaling is already strained (don't buy into the "exponential" hype). And, in any case, the competition will be against the frontier models that will exist in two years.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by majormajor on Sunday May 17, 2026 @12:06PM (#48170143)
> I seriously doubt it. Scaling is already strained (don't buy into the "exponential" hype). And, in any case, the competition will be against the frontier models that will exist in two years.

The big question I'd be asking if I was investing in one of the big players is if those changes are "it can do 99% instead of 97% of the tasks a user will throw at it" (at which point going local and taking back cost control/ownership makes a lot of sense, especially for companies) OR "it will fully replace a human with better output"?

I already don't need Opus for a lot of my tasks and choose instead faster/cheaper ones.

The former is a company that's gonna be trying to sell mainframes against the PC. The latter is a company that is in potentially huge demand, assuming the replaced humans end up with other ways of getting money to still be able to buy stuff in the first place. ;)

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by iwontberude on Sunday May 17, 2026 @12:20PM (#48170320)
Exactly the right argument. Local LLM doesn’t need to outrun the bear (outperform data centers) it only needs to outrun its friend (total cost of ownership).
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by christopherwxyz on Sunday May 17, 2026 @11:54AM (#48170027)
I would readjust your convictions.

We are only 2-4 years away from consumer grade immutable-weight ASICs.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by slashdave on Sunday May 17, 2026 @11:58AM (#48170074)
We are discussing how rapid development has been, and now you want to freeze your model in silicon?
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by nixon_why69 on Sunday May 17, 2026 @12:07PM (#48170153)
Why not have a bunch of SRAM and various operations like "Q4 matmul" in silicon? Model weights and even architectures could still evolve on a platform like that.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by ac29 on Sunday May 17, 2026 @12:56PM (#48170664)
Doesnt "a bunch of SRAM" top out at maybe a few gigs per chip (with zero area used for logic)? You'd need an order of magnitude more to fit even a fairly weak general purpose LLM model.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by throwa356262 on Sunday May 17, 2026 @12:57PM (#48170673)
I belive that is what NPUs are.

The issue is the very huge amount of DRAM and high bandwidth these model require.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by rogerrogerr on Sunday May 17, 2026 @11:59AM (#48170086)
Genuine question from a place of ignorance: what in the silicon pipeline makes it take 2-4years to produce chips with a new model on them? Curious what the process bottleneck is.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by pjc50 on Sunday May 17, 2026 @12:14PM (#48170239)
I think you could get it down to three months between weight changes, if you can encode it in metal layers only. The remaining limits are the fab lead time, and the cost of a metal respin (hundreds of thousands to millions of dollars depending on process).
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by jazzyjackson on Sunday May 17, 2026 @12:08PM (#48170164)
Without being an insider, I imagine that most global fab capacity is contracted out several years in advance.

You might be interested in the tiny tape out project, which guides you through the process of getting your own design etched on silicon. If you only need larger features and not the next gen single digit nanometer stuff, you may not be so supply constrained.

https://tinytapeout.com/

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by happosai on Sunday May 17, 2026 @02:54PM (#48172059)
I think that comment meant it's 2-4 years until local models are good enough that it's worthwhile to burn an ASIC of them. Not that it takes 2-4 years to make an ASIC chip.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by dangus on Sunday May 17, 2026 @12:12PM (#48170221)
If the silicon costs $200-300 and the company throws it away every two years that’s cheaper than a subscription.

Also, how many companies will just buy an M6/M7 MacBook Pro with 32GB+ of RAM in a couple of years and get “free” AI along with the workstation they were going to buy anyway?

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by m101 on Monday May 18, 2026 @03:35AM (#48176532)
This will be interesting. I can see some world where it’s used with consumers, but for the most part I think it will be in the cloud and that would make most sense
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by comfysocks on Sunday May 17, 2026 @12:56PM (#48170666)
> I seriously doubt it. Scaling is already strained (don't buy into the "exponential" hype). And, in any case, the competition will be against the frontier models that will exist in two years.

But even if scaling plateaus for the frontier models, maybe distillation will improve to the point where smaller more manageable models can reach the same plateau. That would be great for local.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by intothemild on Sunday May 17, 2026 @12:15PM (#48170258)
I've spent the last month bringing in a small demo of what the future could be like, running Qwen, Gemma, and Deepseek, behind LiteLLM so we can monitor token usage, and instead of some dumb ass "tokenmaxxing" we're actively trying to get the cost of inference both down, and in-house.

Boss is happy, very happy. We're rolling it out more widely now.

But this is the future.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by stingraycharles on Sunday May 17, 2026 @12:00PM (#48170089)
The economics of local AI just doesn’t make sense. A model like Opus is - supposedly - something like 5T parameters, which is likely something like 3TB of GPU memory.

Local models never reach the % utilization that cloud providers have (80%+), and they’re always going to be much better than local models for this reason.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by lumost on Sunday May 17, 2026 @12:05PM (#48170129)
Capex, opex, quality, and volume are tricky things to balance. On balance, pc/mobile are cheaper to operate than equivalent cloud and on prem deployments.

It’s not unreasonable to suppose that in 2 years time an opus 5 quality model will be etched into silicon for high performance local inference. Then you just upgrade your model every 2-3 years by upgrading your hardware.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by jazzyjackson on Sunday May 17, 2026 @12:16PM (#48170264)
I haven't been following anyone baking models into ASICs, is it not still necessary to pack just as many transistors onto a chip, whether it's an NPU or GPU, ASIC or not you still need to hold hundreds of gigabytes in memory, so how is it cheaper to bake it onto custom silicon than running it on commodity VRAM? (Asking because I don't know!)
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by lumost on Sunday May 17, 2026 @01:34PM (#48171069)
Not my area either! But my understanding is that there are more efficient methods of representing static numbers when you can skip the vram lookup.

https://taalas.com/

Is an example startup in this area claiming 16k tok/s on an asic for llama 8b. Qwen has a 27b model at opus 4.5 quality.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by jazzyjackson on Sunday May 17, 2026 @02:06PM (#48171485)
Neat, thanks for the link
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by majormajor on Sunday May 17, 2026 @12:10PM (#48170198)
Running local applications is less efficient than thin clients to the cloud generally, not just in LLMs. The trick is that you can get to the point where it's effective enough, and affordable enough, that the control and availability factors become dominant.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by stingraycharles on Sunday May 17, 2026 @12:19PM (#48170307)
My point is that you will always get much more value / $ by using cloud based solutions.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by majormajor on Sunday May 17, 2026 @12:44PM (#48170572)
I just don't see how that's different from getting more value by giving all your employees the most stripped-down chromebook-type devices and running everything else in the cloud, than by giving them "proper" laptops with local apps.

It's a measure of a very thin sort of "value/$" that excludes a lot of other things that could be of value to a business, like control, predictability, and availability.

Thin clients have been going away for a long time. The trend has been to continue to push higher levels of compute into ever-smaller and ever-more-portable devices.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by sroerick on Sunday May 17, 2026 @02:13PM (#48171573)
I don't know that this is true. The cloud companies are making money, and inferrence is kind of just "hosting an inferrence server and trying to keep it humming 24/7"

But in many cases self hosted or dedicated boxes are cheaper than cloud.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by adamgordonbell on Sunday May 17, 2026 @10:41AM (#48169366)
Or put another way, the frontier models are very quickly deprecating assets, because of the competition in the market.

They have to keep getting better to stay ahead of each other and open weight.

Which means it's the opposite of a timebomb, the article has it completely backwards, tokens at current level of reasoning will continue to get cheaper.

I'm not sure 'local' will be the end state, as hardware needs are high. But certainly competitive forces tend to push profit margins toward zero.

Extended discussion on this topic:

https://corecursive.com/the-pre-training-wall-and-the-treadm...

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by airstrike on Sunday May 17, 2026 @10:51AM (#48169443)
Well, it's a timebomb for the companies who get paid per token, so the parent is right and TFA is probably wrong
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by vb-8448 on Sunday May 17, 2026 @11:23AM (#48169749)
> within a few years we will be running local models as good as today’s frontier

Unless there isn't some important breakthrough in hw production or in models architecture, it's quite the opposite: bigger, more expensive and more energy-intensive hw is needed today compared to 1 or 2 years ago.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by chermi on Sunday May 17, 2026 @12:04PM (#48170123)
Per frontier token. You're not calculating the cost of a fixed quality asset here. Old hw running non-frontier models will be very valuable. In fact, we have two direct examples: older server gpus actually appreciating and the very obvious fact that not everyone always use MAX FULL EFFORT BEST MODEL no matter what.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by vb-8448 on Sunday May 17, 2026 @04:05PM (#48172735)
Not consumer hw ... and if we speak about local llm we cannot assume most of us can put a rack in their basement

Already today is not possible to run deep seek v4 pro locally, and I cannot imagine that in 2 years we will be.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by evgen on Sunday May 17, 2026 @11:42AM (#48169921)
I can run qwen3.6-27b on a four year-old Macbook Pro that dominates ChatGPT-4o (the frontier model from 2 years ago) and is competetitve against early ChatGPT-5 versions. We are also getting a lot smarter about using and deploying these local models. Your entire AI stack from two years ago would be absolutely crushed by a todays local LLM models and a high-end local inference system when combined with a good modern coding agent.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by vb-8448 on Sunday May 17, 2026 @02:14PM (#48171575)
Today open weights frontier models cannot run locally, unless quantization is used. Deep seek v4 pro require almost 1 TB of RAM in INT4.

I hardly doubt there will be consumer grade HW to run it in 2 years either. And deep seek v4 pro is not even close to OAI or anthropic frontier models.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by ls612 on Sunday May 17, 2026 @11:34AM (#48169842)
As good as today’s frontier. Gemma 4 today is roughly equivalent to the frontier a year and a half ago at gpt 4o tier.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by antisthenes on Sunday May 17, 2026 @11:36AM (#48169869)
What's the cheapest PC you can buy today that will comfortably run Gemma 4 and everything else you want it to run at the same time?

And how many tokens would that buy?

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by ls612 on Sunday May 17, 2026 @11:41AM (#48169911)
I run it on my 4 year old MBP and get 10 tok/s. With the RAM shortage buying anything new today is a nightmare but anyone with a reasonably modern Mac could run it at q6 probably. It is mostly a toy as 4o models weren’t really suitable for real work IMO but at least it won’t ever give me a refusal.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by jazzyjackson on Sunday May 17, 2026 @12:21PM (#48170325)
At 10toks, are you using it interactively or do you submit a prompt and come back to it later? I always thought it would make sense to just do conversations over email, asynchronously, the model can take all the time it needs and get back to me when it has an answer.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by ls612 on Sunday May 17, 2026 @01:34PM (#48171074)
10 tok/s is around the borderline of interactive being good. I did the math and it is mostly bottlenecked by memory bandwidth, so in the future I can expect to run a similarly sized model on my 4090 once it gets retired from gaming service and get ~25 tok/s which will be very usable.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by planb on Sunday May 17, 2026 @12:05PM (#48170128)
If that’s true, then it will be even cheaper to provide them as a subscription. Following your logic, every company would be running their own data centers instead of using cloud providers.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by adrithmetiqa on Sunday May 17, 2026 @12:10PM (#48170195)
I disagree. No one will want to use second rate models when the frontier models reach a specific level of capability. Enterprise will keep paying.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by malfist on Sunday May 17, 2026 @12:12PM (#48170213)
No one? When free means I get 95% of the capabilities of something very very expensive, you bet your bottom dollar many many people will choose free.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by xboxnolifes on Sunday May 17, 2026 @02:17PM (#48171618)
But its not free.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by upcoming-sesame on Sunday May 17, 2026 @04:20PM (#48172852)
not every company can pay for the best engineers in the market, some can only afford to pay for cheaper engineers and it's fine

same with models.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by otterley on Sunday May 17, 2026 @12:01PM (#48170100)
People who are this certain of their predictions should be forced to put real money on them on Kalshi or Polymarket instead of drive-by blowharding on HN.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by watwut on Sunday May 17, 2026 @12:31PM (#48170425)
Meh, having opinioms should imply necessity to gamble on gambling site.

Not even when that site calls itself "market" to create plausible deniality.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by whackernews on Sunday May 17, 2026 @01:03PM (#48170739)
Oooh. You’re hard.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by nijave on Sunday May 17, 2026 @11:01AM (#48169544)
>within a few years

Eventually, we'll see. Frontier models still need some pretty serious hardware which will slowly come down in cost. Smaller models are becoming more capable, which will presumably continue to improve.

I think there's still a pretty big gap, though. Claude estimates Opus 4.6 and GLM-5 need about 1.5Ti VRAM. It puts gpt-5.5 around 3-6Ti of VRAM.

That's 8x Nvidia H200 @ ~$30k USD each. Still need some big efficiency improvements and big hardware cost reduction.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by throw1234567891 on Sunday May 17, 2026 @11:21AM (#48169721)
Or a single mlx cluster if one can find second hand machines somewhere. Difficult to get your hands on today, certainly, but not impossible.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by snovv_crash on Sunday May 17, 2026 @01:33PM (#48171063)
Qwen 3.6 27b is somewhere around Opus 4. It runs on a 5090, a $2k desktop GPU, at reasonable speeds.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by YesBox on Sunday May 17, 2026 @11:24AM (#48169763)
You'd have a point if Cloud ^tm didnt take off into a multi billion dollar industry.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by wolttam on Sunday May 17, 2026 @11:21AM (#48169729)
There's still going to be plenty of use-case and demand for frontier models running across hundreds or thousands of GPUs. It's just not going to be in the current shape - certainly not accessed by the general public for rote business tasks.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by jmount on Sunday May 17, 2026 @12:43PM (#48170566)
I think this is a good under-represented point. Again and again things that could only run on a mainframe get ported to the personal device level. However it looks like the campaign to eliminate the PC (by pre-buying all RAM) is the counter-stroke.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by himata4113 on Sunday May 17, 2026 @01:00PM (#48170704)
This is wrong because local models are very expensive, just as expensive as the frontier.

It would cost me $300 in normal deepseek v4 pricing (non discounted) PER DAY, but I get it all for $500 worth of subscriptions.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by nozzlegear on Sunday May 17, 2026 @02:22PM (#48171671)
Why are you paying $300/day to run a local model? The whole point is that you run them on a machine you already own.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by himata4113 on Sunday May 17, 2026 @03:02PM (#48172151)
None of the models advanced enough to replace frontier will be able to run on your machine for any forseeable future or at a reasonable speed. 5tok/s is not acceptable.

To run deepseek v4 class model, you would need to spend $120k just in gpus.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by aleqs on Sunday May 17, 2026 @01:02PM (#48170737)
Hard agree - the benefits of local/self-hosted models are not just hardware/cost (it might be more expensive at the moment), but what you get in exchange is unnerfed/unstupified models, full cost/usage transparency, optimized/specialized models, privacy/security, etc.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by WarmWash on Sunday May 17, 2026 @02:24PM (#48171709)
Linux in year 2000 vibes...still waiting to get off windows 26 years later
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by claysmithr on Sunday May 17, 2026 @03:13PM (#48172259)
I agree. The AI bubble is going to pop, people will move to local models, and the datacenters will be abandoned
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by voxleone on Sunday May 17, 2026 @04:37PM (#48172983)
I can only hope that you'll be right someday. As of now, an RTX 3090 struggles to run most of the good local models.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by czep on Sunday May 17, 2026 @05:10PM (#48173227)
The economic question is whether the average company will have the time or talent to roll their own models instead of eating the cost increases. The firms in question are exactly the same that have already decimated their teams. Can they so quickly pivot to self-hosted models if their AI workloads suddenly cost them 10x more? I bet most will simply start shoveling themselves deeper.
Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by tsycho on Sunday May 17, 2026 @06:58PM (#48173899)
A lot of home computers are capable (with a large margin) to run a large amount of self-hosted services (eg: jellyfin, immich, minecraft, plex, karakeep, ... whatever people want to use).

And yet, less than 0.01% of the population (made up number, but I am more likely to be overestimating than underestimating) do so.

Running local models to do real work is likely to be another niche hobby.

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by throaway198234 on Sunday May 17, 2026 @08:00PM (#48174252)
^^^^^^^^^^^

AI is the future operating system of every computer everywhere

Re:Every AI Subscription Is a Ticking Time Bomb for Enterprise by Ferret7446 on Monday May 18, 2026 @01:51AM (#48175985)
That's why cloud died out and everyone is running their own servers right?