News for hackers. Stuff that matters.
Re:IBM CEO says open weight models are now irresistibly good... by glassresearch on Monday October 26, 2026 @12:13AM (#50239036)
I'm not saying your approach is wrong - but for my workloads, it makes a lot more sense to just pay for a decade (!!!) of Claude Code 20x Max.
Re:IBM CEO says open weight models are now irresistibly good... by calmingsolitude on Monday October 26, 2026 @12:32AM (#50239151)
LLM subscriptions are heavily subsidized. I’d be surprised if Claude 20x costed the same after a decade.
Re:IBM CEO says open weight models are now irresistibly good... by distances on Monday October 26, 2026 @03:25AM (#50240273)
Inference prices are going only down. It could be that some top subscription costs more in a decade, but I fully believe a full array of competent models will be extremely cheap.
Re:IBM CEO says open weight models are now irresistibly good... by Gareth321 on Monday October 26, 2026 @03:40AM (#50240392)
The inference is profitable. It’s the training which is really expensive and not being reflected in the subscription prices.
Re:IBM CEO says open weight models are now irresistibly good... by latentsea on Monday October 26, 2026 @03:27AM (#50240288)
Doesn't deepseek make $38 per GPU-hour? As in they can actually serve profitably?
Re:IBM CEO says open weight models are now irresistibly good... by petu on Monday October 26, 2026 @04:16AM (#50240692)
Deepseek doesn't have subscriptions, you can only pay per token / API prices.
Re:IBM CEO says open weight models are now irresistibly good... by pimeys on Monday October 26, 2026 @04:58AM (#50241007)
Yes but the API prices are very low. So much that it is hard to spend over 20 dollars per month as an individual no matter how much you use it.
Re:IBM CEO says open weight models are now irresistibly good... by latentsea on Monday October 26, 2026 @06:55AM (#50241803)
How are their API prices compared to others?
Re:IBM CEO says open weight models are now irresistibly good... by jacquesm on Monday October 26, 2026 @12:38AM (#50239188)
If I want to go out of business that is what I would do. Most of the data I have here could never be shipped to a service provider, I'd be in lawsuits for the next decade or so and die bankrupt.
Re:IBM CEO says open weight models are now irresistibly good... by greesil on Monday October 26, 2026 @12:48AM (#50239262)
Okay man I've got to ask, what kind of data are we talking about? Legal? Medical?

I've heard that there are legal software platforms like Clio that offer AI products, not sure what just eavesdropping on some lawyers at the bar.

Re:IBM CEO says open weight models are now irresistibly good... by jacquesm on Monday October 26, 2026 @12:51AM (#50239278)
http://www.modularcompany.com/

Companies pretty much give us the full overview of what they have from a technical perspective (and often much more than that), we sign NDAs with investors that have serious teeth in them. The data they share with us would not be disclosed under any other circumstance and even our reports are shared only with the clients.

It's a nice little business. I'm not going to risk offending my customers for convenience or to save some money.

Re:IBM CEO says open weight models are now irresistibly good... by greesil on Monday October 26, 2026 @01:29AM (#50239511)
Holy fuck you have a lot of karma
Re:IBM CEO says open weight models are now irresistibly good... by disgruntledphd2 on Monday October 26, 2026 @02:34AM (#50239893)
He's been here for a loooonnnggg time. I'm pretty sure I found HN through one of his blog posts.
Re:IBM CEO says open weight models are now irresistibly good... by 4gotunameagain on Monday October 26, 2026 @05:07AM (#50241070)
You can estimate the amount of time by the domains he's been squatting all these years.
Re:IBM CEO says open weight models are now irresistibly good... by dekhn on Monday October 26, 2026 @02:36PM (#50248748)
Isn't he the guy who posted about his LEGO-sorting computer vision project? That post alone taught me things that increased my career opportunities significantly.
Re:IBM CEO says open weight models are now irresistibly good... by robotmay on Monday October 26, 2026 @05:50AM (#50241362)
On a tangent: this is a really good website. I understood what you do and how you do it by reading down one page. There's a lot of projects/companies I immediately tune out from because I can't actually figure out what they do amongst all the buzzwords, animations, and excess scrolling in every dimension.
Re:IBM CEO says open weight models are now irresistibly good... by Asyakguen on Monday October 26, 2026 @04:58AM (#50241006)
That NDA case is the real argument, more than price. Token prices keep dropping anyway, but "this data can never leave the building" doesn't change with a price cut ;)
Re:IBM CEO says open weight models are now irresistibly good... by tjwebbnorfolk on Monday October 26, 2026 @12:42AM (#50239220)
The bet we (self-hosters) are making is that once these companies IPO, they will be forced to do whatever they have to do to make money. i.e. price increases, more throttling of subscription non-usage-based accoutns, etc.

They are subsidizing all of us to the tune of $billions in losses per month. That can't go on for a decade. Probably not even for another 18 months.

I still use all the tokens Claude will give me on the max plan. But I also run deepseek v4 locally for overflow and for... contingencies.

Re:IBM CEO says open weight models are now irresistibly good... by lelanthran on Monday October 26, 2026 @04:53AM (#50240971)
> The bet we (self-hosters) are making is that once these companies IPO, they will be forced to do whatever they have to do to make money. i.e. price increases, more throttling of subscription non-usage-based accoutns, etc.

But, there is no need to make that bet - when prices increase, I'd look into renting a previous-gen GPU on a cloud provider. If the online GPU providers raise prices, then I would look into purchasing for on-site.

Until that happens, I'm not making any bets, because if all the token providers go bust GPU rentals are going to fall in price, and if they don't go bust after raising prices, I can still get years of GPUs for self-hosting at the price of a self-owned rig.

Re:IBM CEO says open weight models are now irresistibly good... by tjwebbnorfolk on Monday October 26, 2026 @12:16PM (#50246307)
Well there are risks to being part of a crowd that's all compelled to make the same decision at the same time. Certain risks can only be transferred, not reduced in absolute terms.

It's not economical today to run my own hardware, but the insurance is worth it to me in the sense that if my bet is wrong, I'm still okay with paying extra for the protection.

Re:IBM CEO says open weight models are now irresistibly good... by benoau on Monday October 26, 2026 @12:49AM (#50239270)
If you know you will pay Claude a few thousand bucks a year a decade adds up too, owning the hardware could just as well end up cheaper because you can sell it or as others have alluded the price to rent could get much worse.
Re:IBM CEO says open weight models are now irresistibly good... by dylan604 on Monday October 26, 2026 @01:10AM (#50239413)
Would you really expect to be using the same hardware over the course of that decade? One of the biggest drawbacks to using on-prem hardware is keeping up with the upgrades. Leasing upgrade cycles are a thing for businesses, but for someone at home it seems like later in the decade the local system will be showing its age.
Re:IBM CEO says open weight models are now irresistibly good... by jacquesm on Monday October 26, 2026 @01:38AM (#50239553)
The 3090's I have would sell for more than new today... for the next couple of years or so it will be pretty bad on the hardware front. Until there is more RAM manufacturing capacity and NV has a credible competitor I don't think it will get better.
Re:IBM CEO says open weight models are now irresistibly good... by jacquesm on Monday October 26, 2026 @12:51AM (#50239276)
Heh, the hardware I bought I could easily sell for 2x so you may actually end up making money. Though I hope that that won't last.
Re:IBM CEO says open weight models are now irresistibly good... by segmondy on Monday October 26, 2026 @01:04AM (#50239362)
My local model never says no. I never get banned. If I want to do security stuff, reverse engine, there it goes! Do I wanna chat about politics? No problem! Do I want it to analyze medical stuff? I don't get to hear that it's not a doctor and I should call 911. Do I want to explore legal docs, no problem! There are lots of things that cloud models will say no to! I read that don't even agree to help with advanced biology and chemistry topic much. I also never run out of tokens, no 5 hr limit. I can run it 24/7. I don't run out of tokens. I can run 16 sub agents at the same time, non stop! 1-4 billion tokens a month? No problem. Any agent harness I want! No restrictions. Privacy all day. My own true system prompt, not something someone has determined, no filters. Furthermore, there are many open weight cloud models that don't have equivalent online. For a while before cloud models got good at math, the best model at math was DeepSeekv2 Math and there was no API. The only way to use it was to run local. All sorts of audio models, image/video models, etc. No multiple subscription needed. Private UI for my family, kids included (I can guard what kids infer). Got a friend that really needs some tokens? I give em access through private VPN. Telling someone that they are paying for decades of Claude for local AI is like telling someone they are paying for decades of public transport when they buy a car. Sure, you are right, but ...
Re:IBM CEO says open weight models are now irresistibly good... by latentsea on Monday October 26, 2026 @03:26AM (#50240277)
> My local model never says no. I never get banned.

So much this. Plus everything else you mentioned. The freedom and control is real.

Re:IBM CEO says open weight models are now irresistibly good... by Gareth321 on Monday October 26, 2026 @03:40AM (#50240385)
Also no political censorship, and it doesn’t report you to the police when it suspects you did something illegal. That really happened, by the way.

Now I want a local LLM server :( $25k is just so much money.

Re:IBM CEO says open weight models are now irresistibly good... by cft on Monday October 26, 2026 @04:31AM (#50240808)
MacBooks with large RAM and optimized for local models should be Apple's medium term goal.
Re:IBM CEO says open weight models are now irresistibly good... by Gareth321 on Monday October 26, 2026 @06:26AM (#50241622)
It's a good first step. They're not great but they kind of work for many tasks. I think the small models can be useful in context-specific tasks but the friction is high right now because the user is forced to determine whether Siri is good enough, or a better LLM is required per task. Ideally Siri would be smart enough to intelligently route. Apple claims that's how it works but even their cloud processing is far behind frontier models.

Worryingly, they're taking a walled garden approach, and they don't want to expose user data to competitors. After the monumental failure of Siri over more than a decade, I don't trust Apple to execute competently on AI.

Re:IBM CEO says open weight models are now irresistibly good... by _ache_ on Monday October 26, 2026 @05:57AM (#50241407)
I'm building a local LLM server myself. For about 8k€ you can have 170tps, ~6000 prefill.

A lot more with Haiku equivalents (Qwen3.8 27B) or gemini. It's not frontier, but not that far away.

Re:IBM CEO says open weight models are now irresistibly good... by Gareth321 on Monday October 26, 2026 @07:08AM (#50241886)
I would love to hear more about your setup. Which hardware and models? How much context can it handle? How does it compare to frontier models?
Re:IBM CEO says open weight models are now irresistibly good... by _ache_ on Monday October 26, 2026 @08:16AM (#50242378)
I mostly run Qwen3.8 Flash Next on 2xR9700 (but Intel Arc Pro B70 may be better currently, or 4x3090 is even better but out of budget, 32Gio Nvidia GCs on ebay are also usable). I have 128Gio DDR4 (3200Hz), but 64Gio is enough, even 32Gio maybe, I just reuse it from an old project.

You may need a AMD EPYC processor and associated motherboard, don't forget a good PSU at least 1200W (300W processor + 400W by card to support spikes of consumption). On some setup (~15k€) you can use 4 CG to go up to 128G of VRAM.

I'm not using a EPYC processor for now, so I get 170tps on decode but very slow prefill, the bottleneck is the PCIe exchange between the cards, I need to setup P2P (it means what you think it means, for GCs).

My setup supports full context (2*18 bytes so 256K), I didn't setup YaRN to get 1M of context (2*20 bytes), but it's an option with 128Gio of RAM, the visual component is extracted from RAM so a little slower. I'm not using multiple agents but some people does, it's god like with 4 graphics cards, on 2 only, it's relatively quick but limited to single user and up to 3 to 4 concurrents requests only.

It's not frontier, but look AA of QFN, it's good enough. DeepSeek V4.1 can run on my setup, but it's slow. On dual B70, there is some PoCs to make it usable (>60tps). It improves every days.

If you need more information, look for local setup users on the Web. There is a lot of people sharing information about benchmarks and setups. L80 is my reference for now.

Re:IBM CEO says open weight models are now irresistibly good... by ashkankiani on Monday October 26, 2026 @09:25AM (#50243260)
Why do you keep writing Gio for gigabytes? I guess maybe GiB for gibibyte, but Gio is new to me
Re:IBM CEO says open weight models are now irresistibly good... by _ache_ on Monday October 26, 2026 @10:11AM (#50244039)
I'm sorry. Yeah Gio is only in french, I forgot to translate the unity.

octet => byte, https://fr.wiktionary.org/wiki/Gio

Re:IBM CEO says open weight models are now irresistibly good... by jacquesm on Monday October 26, 2026 @10:29AM (#50244364)
You're Blue Book compliant ;)
Re:IBM CEO says open weight models are now irresistibly good... by ashkankiani on Monday October 26, 2026 @10:45AM (#50244641)
oh neat, I didn't know that. makes sense, thanks
Re:IBM CEO says open weight models are now irresistibly good... by ThunderSizzle on Monday October 26, 2026 @06:22AM (#50241592)
If you already have a machine, throwing an r9700 in it is one of the cheapest ways to get qwen3.8 27B Q6 with 100k+ context working.

Sadly, the r9700 has gone up in price, but it's still the cheapest route. I use It on an old x99 Linux desktop and it achieves 50 tps decode and 1000+ tps prefill.

Re:IBM CEO says open weight models are now irresistibly good... by Gareth321 on Monday October 26, 2026 @07:07AM (#50241882)
Thanks for the tip! I've already got a home server set up. Maybe this is my next AI purchase. Still, US$2,600 here in Europe is not cheap :-/
Re:IBM CEO says open weight models are now irresistibly good... by jacquesm on Monday October 26, 2026 @10:28AM (#50244346)
That's true. Now let's reverse things a bit, we go back five years in time and I give you 'magic in a box' for $2600. Would you buy it?
Re:IBM CEO says open weight models are now irresistibly good... by ThunderSizzle on Tuesday October 27, 2026 @06:11AM (#50256850)
I was hesitant, so I went with a hassle-free return route when I bought mine.

My stack uses a local llamacpp ROCM build, modified by https://github.com/stew675/llama-cpp-rdna-boosts

The build uses my machine's ROCM and I just link it instead of embedded all that in an image, like the published ones do.

Then deploying it via docker compose to my local machine. The models I want to run are stored in a docker volume, so it's easy to switch the model while keeping the same build.

The biggest negative for me is that I don't think it's enough to do parallelization through llamacpp (I'd prefer to work on one thing faster than two things slowly). I haven't gotten vllm working yet. Also, once you have one, you constantly start thinking about a second card...

Re:IBM CEO says open weight models are now irresistibly good... by charcircuit on Monday October 26, 2026 @06:25AM (#50241617)
>and it doesn’t report you to the police when it suspects you did something illegal

This is an alignment issue. OpenClaw with a local LLM with an email you setup for it could contact the police.

Re:IBM CEO says open weight models are now irresistibly good... by Gareth321 on Monday October 26, 2026 @06:54AM (#50241792)
It will not do that unless you specifically teach it to do that, or you use an Anthropic model. The issue in this case was Anthropic's safety monitoring built into the model. They record everything and flag potential illegal activity for human review. This occurs even over the API. I do not know if OpenAI does this, but I would not be surprised. I think the Chinese models are far less likely to do this (unless one resides in China).
Re:IBM CEO says open weight models are now irresistibly good... by ceejayoz on Monday October 26, 2026 @10:21AM (#50244190)
> It will not do that unless you specifically teach it to do that…

Sorry, you're claiming local LLMs are somehow deterministic and never show unexpected behavior?

Re:IBM CEO says open weight models are now irresistibly good... by Gareth321 on Monday October 26, 2026 @02:03PM (#50248156)
I’m suggesting it’s highly unlikely it will call the FBI to dob you in. That affirmatively trained behaviour, not something one should expect to accidentally happen.
Re:IBM CEO says open weight models are now irresistibly good... by ceejayoz on Monday October 26, 2026 @02:26PM (#50248586)
Why would you assume an AI model's training set doesn't have things like "you should report crimes to law enforcement" in it?
Re:IBM CEO says open weight models are now irresistibly good... by Jnr on Monday October 26, 2026 @06:43AM (#50241725)
No political censorhip? Try asking Chinese models about Taiwan.

Some censorhip and bias towards certain ideology is baked into those models.

While it is not a problem for programming tasks, I would never use it for certain other tasks.

Re:IBM CEO says open weight models are now irresistibly good... by Gareth321 on Monday October 26, 2026 @07:04AM (#50241863)
> No political censorhip? Try asking Chinese models about Taiwan.

Self-hosting means being able to use abliterated models where the superficial instructions like not describing Taiwan as independent are burned off.

Censorship and bias are related but not synonymous. I agree that it's harder to remove political bias when the training data includes it. We know many of the frontier models are heavily trained on Reddit, for example, [which likely explains why we see such heavy left wing bias.](https://trakkr.ai/bias/findings)

More citations:

https://www.gsb.stanford.edu/faculty-research/working-papers... (article: https://news.stanford.edu/stories/2025/05/ai-models-llms-cha...)

https://www.tandfonline.com/doi/full/10.1080/29974100.2026.2...

https://ieeexplore.ieee.org/document/10817610

https://www.washingtonpost.com/technology/interactive/2026/0...

https://link.springer.com/article/10.1007/s11127-023-01097-2

Re:IBM CEO says open weight models are now irresistibly good... by Jnr on Monday October 26, 2026 @07:37AM (#50242061)
All good points, but I am not sure who I trust less. Chinese companies or random people on Hugging Face. :) It is still a black box. And I have a feeling I could not abliterate those models on my own hardware.

I am not saying I would not use any of them at all, but I would avoid doing certain tasks with them. And with the testing I have done with abliterated models in the past, it is not 100% successful and it can still hit the same walls as original model.

Re:IBM CEO says open weight models are now irresistibly good... by Gareth321 on Monday October 26, 2026 @08:52AM (#50242763)
I agree. I think the future for privacy conscious people is going to be an adversarial open weights model, where competing models (ideally from opposing superpowers and labs) critique each other's work to find consensus. Or at least alert for suspicious behaviour.
Re:IBM CEO says open weight models are now irresistibly good... by xeyownt on Monday October 26, 2026 @07:51AM (#50242181)
So... the Super Intelligence is Democrat?
Re:IBM CEO says open weight models are now irresistibly good... by scotty79 on Monday October 26, 2026 @08:30AM (#50242531)
After years of trump any intelligence is a democrat.
Re:IBM CEO says open weight models are now irresistibly good... by pluralmonad on Monday October 26, 2026 @10:25AM (#50244281)
The only winning move...
Re:IBM CEO says open weight models are now irresistibly good... by scotty79 on Monday October 26, 2026 @10:30AM (#50244385)
Is to deprive yourself of the slightest chance of winning?
Re:IBM CEO says open weight models are now irresistibly good... by schiffern on Monday October 26, 2026 @08:37AM (#50242604)

  >abliterated models where the superficial instructions like not describing Taiwan as independent are burned off
Does this help with issues like including security vulnerabilities when you say you're in Tibet? I understood that this was burned in at the training level, not just a layer of censorship added at the last minute.

https://www.crowdstrike.com/en-us/blog/crowdstrike-researche...

Re:IBM CEO says open weight models are now irresistibly good... by Gareth321 on Monday October 26, 2026 @08:49AM (#50242731)
It could, but it's an inexact science. A little bit like tampering with the brain. Even a surgical procedure removing a tiny tumor can interfere with critical pathways. Usually abliterated models end up almost exactly as capable in benchmarks, but it requires the person doing the abliteration to anticipate or know about the censorship. In the case of Chinese model censorship it is believed that it is post-training, so the underlying data input is not the issue. It's how the model has been trained to use that data. Specifically, abliteration usually targets the refusal behaviour. I would guess that this example could be significantly improved, but I have not seen any abliteration attempts re this specifically.
Re:IBM CEO says open weight models are now irresistibly good... by deepersprout on Monday October 26, 2026 @10:25AM (#50244278)
> No political censorhip? Try asking Chinese models about Taiwan.

In open models you can "uncensor" them: https://arxiv.org/abs/2406.11717

Re:IBM CEO says open weight models are now irresistibly good... by InsidiousWombat on Monday October 26, 2026 @08:26AM (#50242493)
renting GPU off cloud is also an option
Re:IBM CEO says open weight models are now irresistibly good... by ali_chherawalla on Thursday October 29, 2026 @05:15AM (#50286374)
You may not need $25k. Off Grid AI runs open models on the desktop/laptop you already have and picks ones that fit its memory, free and offline. Bigger ones run on your desktop and you can use them from your phone without any cloud servers in between. either your wifi or a tailnet like tailscale
Re:IBM CEO says open weight models are now irresistibly good... by jacquesm on Monday October 26, 2026 @01:06AM (#50239371)
Exactly, that's the spirit. I pool the resources with some friends as well.
Re:IBM CEO says open weight models are now irresistibly good... by Forgeties79 on Monday October 26, 2026 @04:42AM (#50240891)
I think both of y’all glossed over the $25k of it all pretty quickly. This isn’t remotely an option for anyone.
Re:IBM CEO says open weight models are now irresistibly good... by solyin on Monday October 26, 2026 @06:13AM (#50241527)
AnyONE. But if you got some friends, and they got friends, then that $25k becomes smaller and stomachable. My favorite rumor out of LA is groups of friends with a Lamborghini Urus that they timeshare. No idea if it's true, but if car people can get together to buy/lease a car that's MSRP is $345k, I think $25k should be an option for anyone with friends. (Hypothetically. I've read a lot about this friend thing on the Internet!)
Re:IBM CEO says open weight models are now irresistibly good... by sailfast on Monday October 26, 2026 @07:17AM (#50241940)
But I don’t need my LLMs part time? And we all live in the same time zone.

And what happens when the girl at the bar figures out that you TIMESHARE that Lambo?

Re:IBM CEO says open weight models are now irresistibly good... by xeyownt on Monday October 26, 2026 @07:55AM (#50242219)
You timeshare the girl?

Ok taking the :door:...

Re:IBM CEO says open weight models are now irresistibly good... by fcarraldo on Monday October 26, 2026 @09:04AM (#50242932)
Given all of the discourse about what’s happened at Cornell, you really thought this was a worthwhile comment to make?

This sort of casual misogyny is at the root of the problem. Shame on you.

Re:IBM CEO says open weight models are now irresistibly good... by Forgeties79 on Monday October 26, 2026 @07:05AM (#50241875)
If 10 people are sharing it the speed advantages suddenly drop drastically. That’s $2500 a head. 5 would be $5k. Then all the rules and such you need in place

This is all this stuff that dries people to pay for frontier models. Even running modest models requires a few thousand, unless you just want really lean ones. People don’t need frontier but it’s not cheap to effectively run, say, a 16gb Qwen 3.6 or 3.8 in the 30b range. Ram is like $15/gb and GPU’s with 16gb vram are hardly cheap.

Re:IBM CEO says open weight models are now irresistibly good... by wtch98 on Monday October 26, 2026 @09:46AM (#50243615)
In the 90s we'd pay $5k a head (inflation adjusted - about $2k then) for a 486 with 80MB of disk space
Re:IBM CEO says open weight models are now irresistibly good... by Forgeties79 on Monday October 26, 2026 @06:17PM (#50251889)
Yes 30-40 years ago when home computers were still a pretty new technology you paid a lot of money just to have them.
Re:IBM CEO says open weight models are now irresistibly good... by wtch98 on Tuesday October 27, 2026 @06:00AM (#50256751)
And all sorts of companies, from small businesses to multinationals, and a fair few people at home, paid it.

Businesses were quite happy to spend $5k on technology for a worker if it makes things better, why wouldn't they today.

Re:IBM CEO says open weight models are now irresistibly good... by Forgeties79 on Tuesday October 27, 2026 @12:10PM (#50261382)
Unless I read it wrong, it sounds like this guy did not get it paid for by his employer - he runs multiple companies and is an angel investor with clearly deep pockets - and I have people talking about it being an investment in their careers. This is not a straightforward business expense conversation IMO. Otherwise yes I do agree with you.
Re:IBM CEO says open weight models are now irresistibly good... by dekhn on Monday October 26, 2026 @02:35PM (#50248726)
In 1993-94, I upgraded the RAM on my home 486, which started with 4MB. I doubled that to 8MB for $250, then to 16MB for another $250, and finally to 32MB (the max for the motherboard) for another $250. At the time, I was a poor undergraduate and had to give up other pleasures for these expenses. If I estimate correctly it's about $30K/gigabyte (only high end servers at the time had a gigabyte, and they would often cost $100K or more).

The lesson it taught me is that swapping is slow and that if it happens during your software development, you're much less productive. When I finally maxxed out the RAM on my machine, I could run linux + emacs + X11 + bash + g++ all at the same time, making me about 10X more productive. It's an investment and lesson that has helped me build my career over decades.

Re:IBM CEO says open weight models are now irresistibly good... by segfaltnh on Monday October 26, 2026 @05:21AM (#50241186)
Its a drop in the bucket for many businesses though.
Re:IBM CEO says open weight models are now irresistibly good... by bityard on Monday October 26, 2026 @08:21AM (#50242432)
You see it as money spent, others might see it as an investment. OP doesn't necessarily say exactly what they use it for, but if education is the goal, $25k and spending your evenings learning about AI theory and tooling is way cheaper than a university degree and can easily result in millions more dollars in lifetime earnings given the current early state of the technology.

Besides, I regularly see people here in HN spend a LOT more than that on hobbies and depreciating assets. A few years back, people with coastal salaries and tech jobs were tripping over themselves to spend $100k on a certain brand of trendy electric car.

Re:IBM CEO says open weight models are now irresistibly good... by Forgeties79 on Monday October 26, 2026 @06:15PM (#50251882)
I can’t “invest” $25k into an LLM. The vast majority of people can’t. And I’m not spending $5k to timeshare it with 4 other people or whatever lol I currently make do with a 9070 + 32gb of ram.
Re:IBM CEO says open weight models are now irresistibly good... by dekhn on Monday October 26, 2026 @02:31PM (#50248666)
Anyone? I know many people in Silicon Valley, Boston, Seattle, New York, and elsewhere who would consider that a reasonable expense if it kept them viable in the job market for a few more years.

My entire career has been based on purchasing the most hardware I could afford at home and using it near its peak. I've given up other life pleasures (dinners out, nicer car/housing) several times to max out my computational capacity.

Also hardware like this, if chosen carefully, often has a viable lifetime over 5+ years

Re:IBM CEO says open weight models are now irresistibly good... by Forgeties79 on Monday October 26, 2026 @09:32PM (#50253645)
I’m not spending $25k to “stay in the job market.” I come from film production, I’m no stranger to owning my own hardware. But that’s ridiculous. And at least people rented my gear.
Re:IBM CEO says open weight models are now irresistibly good... by jack_pp on Monday October 26, 2026 @06:13AM (#50241525)
isn't it the same using the same model you host locally but through a cloud provider? does the $ math favour a local setup then?
Re:IBM CEO says open weight models are now irresistibly good... by segmondy on Monday October 26, 2026 @10:54AM (#50244814)
Cloud models sometimes get degraded. You can trust your local model and know what you are running.
Re:IBM CEO says open weight models are now irresistibly good... by jack_pp on Monday October 26, 2026 @11:54AM (#50245895)
seems like a stretch, especially when you could choose from 10 of them if u feel they're dropping off.

Also, it's not like you're running any full models on local without shelling out 20k+

Re:IBM CEO says open weight models are now irresistibly good... by charcircuit on Monday October 26, 2026 @06:23AM (#50241594)
>I don't get to hear that it's not a doctor and I should call 911.

Local models can do this and with tool use they can even call emergency services for you on your behalf.

Re:IBM CEO says open weight models are now irresistibly good... by ceejayoz on Monday October 26, 2026 @06:29AM (#50241643)
> My local model never says no.

> I don't get to hear that it's not a doctor and I should call 911.

https://en.wikipedia.org/wiki/Darwin_Awards

Re:IBM CEO says open weight models are now irresistibly good... by mik1998 on Monday October 26, 2026 @07:04AM (#50241865)
It's pretty easy for a reasonable person to determine when to call 911 and when to ask an LLM. Typically when I ask an LLM, it is for something I am not willing to escalate further at the moment.

Anthropic has a significant interest in reducing liability, which in this case means mostly blocking this use case.

Re:IBM CEO says open weight models are now irresistibly good... by ceejayoz on Monday October 26, 2026 @09:51AM (#50243711)
> It's pretty easy for a reasonable person to determine when to call 911…

Unfortunately, it's fairly tough for someone to determine if they are themselves a reasonable person.

A model that only ever says yes is a cognitohazard, IMO. Like being a billionaire without the money to insulate you from the consequences.

Re:IBM CEO says open weight models are now irresistibly good... by pluralmonad on Monday October 26, 2026 @10:33AM (#50244423)
Saying yes is not equal to "provides the requested information." If you use models for affirmation instead of information, this is a more reasonable concern.
Re:IBM CEO says open weight models are now irresistibly good... by ceejayoz on Monday October 26, 2026 @10:42AM (#50244578)
"I don't get to hear that it's not a doctor and I should call 911."

Sometimes, the requested information should be accompanied with a note like that.

Re:IBM CEO says open weight models are now irresistibly good... by segmondy on Monday October 26, 2026 @10:53AM (#50244790)
Have you heard the cost of health insurance and seeing a doctor? Some people don't see going to hospital as an option. Better for them to have a local model than no options. But let me tell you how I use it for health, I take all my health records, lab results, every thing, I throw it in there for a second opinion, it explains things to me, it suggests questions I should ask on follow up, possible options, etc.
Re:IBM CEO says open weight models are now irresistibly good... by ceejayoz on Monday October 26, 2026 @11:00AM (#50244938)
> Have you heard the cost of health insurance and seeing a doctor?

Yes. I've also heard the cost (death) of people falling into pseudosciency rabbit holes.

Re:IBM CEO says open weight models are now irresistibly good... by sharmajai on Monday October 26, 2026 @09:22AM (#50243201)
A cloud model can also get neutered anytime and/or start using more tokens while charging you the same or more per token.
Re:IBM CEO says open weight models are now irresistibly good... by JTbane on Monday October 26, 2026 @10:24AM (#50244243)
The security research is probably the biggest reason to use local models as it seems like the big service providers will ban you for life if you even hint at doing that.
Re:IBM CEO says open weight models are now irresistibly good... by dekhn on Monday October 26, 2026 @02:22PM (#50248488)
I agree the local models have many nice properties, but I would not be surprised if people start to recognize that Gemini pointing you at medical services instead of attempting to diagnose you is actually a good thing in many cases (and Gemini often does do some diagnosis anyway).
Re:IBM CEO says open weight models are now irresistibly good... by elevation on Monday October 26, 2026 @07:32PM (#50252683)
> My local model never says no

In case a future user wants to know what a 2026 model would refuse to do, I needed to rotate dozens of passwords last night both on the server and on the clients. When it was time to verify that the rotated credentials worked, I asked for a script that would rate limit the credential checks. Claude/opus5 (opus!!) refused and stopped in its tracks.

In another instance, Claude was making great headway bringing a legacy Windows app up in WINE until it’s saw “auth” somewhere and shut down. In the shutdown state, Claude asked me to reword the prompt to see if I could get it to act on less dangerous instructions. But when I reworded my request, narrowed the scope to be clearly unthreatening, or even redirected the model, it’s still rejected me. I suspect Anthropic uses the less suspicious rephrasing as training material to increase their rejection rate.

Re:IBM CEO says open weight models are now irresistibly good... by ffsm8 on Monday October 26, 2026 @02:38AM (#50239917)
If you compare to API prices the value difference is not as big.

The anthropic sub is likely still subventioned with VC money. So this may not even be available in 2 years.

But yeah, with current sub prices/budget... It'd stick with it too

Re:IBM CEO says open weight models are now irresistibly good... by ffsm8 on Monday October 26, 2026 @02:41AM (#50239939)
If you compare to API prices the value difference is not as big.

The anthropic sub is likely still subventioned with VC money. So this may not even be available in 2 years.

But yeah, with current sub prices/budget... It's hard to go with anyone else unless you're rich.

Re:IBM CEO says open weight models are now irresistibly good... by reacharavindh on Monday October 26, 2026 @04:05AM (#50240595)
I was thinking the economics of it the same way.. if the AI companies are burning debt and VC money to give me cheaper tokens why not take them for it? I can always rent a GPU server for those personal, private AI needs.. heck even run a decision model to route them to the rented GPU servers either on demand or scheduled for few hours in a day…

It doesn’t have to be black or white…

For my toy projects, adventures, learning exercises, and throwaway MVPs I will prefer the subscription plans… they are cheap for what they offer today.

Re:IBM CEO says open weight models are now irresistibly good... by tourmalinetaco on Monday October 26, 2026 @04:12AM (#50240654)
Yeah, the way I see it short term is take the free/subsidized tokens while we can and the further out we go the cheaper compute will get for local setups.
Re:IBM CEO says open weight models are now irresistibly good... by thaanpaa on Monday October 26, 2026 @04:18AM (#50240714)
From this article alone it should be obvious that the long-repeated myth about frontier inference being subsidized is just that: a myth.

It's training and tuning new models that cost them money. And that's why they are trying to scare people into thinking AI will kill us all in the next 5 years; they want bans on the open-weight models because they know they won't be able to compete with them.

Re:IBM CEO says open weight models are now irresistibly good... by pimeys on Monday October 26, 2026 @04:56AM (#50240993)
A few days ago we got the news that DeepSeek makes 38 dollars profit per card with their off peak pricing (in the inference server release discussion).

If that is true, either the American labs are not optimizing correctly or they make a ton of money with inference.

Re:IBM CEO says open weight models are now irresistibly good... by GeekyBear on Monday October 26, 2026 @05:13AM (#50241112)
DeepSeek designs models that require less hardware resources for inference.
Re:IBM CEO says open weight models are now irresistibly good... by bityard on Monday October 26, 2026 @08:52AM (#50242765)
There are lots of reasons Chinese AI companies will always be cheaper than Western AI companies, but chief among them is that China is pouring ludicrous amounts of money into new energy generation projects.

Hydro, wind, solar, nuclear, coal, they are building it all, as we speak, at least 10x more than the US. They are aiming to not just be energy independent, but energy dominant.

You can't now (and certainly won't in the future) be able to run an AI provider in the West as profitably as you can in China.

Re:IBM CEO says open weight models are now irresistibly good... by MarcusE1W on Monday October 26, 2026 @04:30AM (#50240798)
Keep in mind that the prices of the big AI companies are currently mostly loss makers. Their plan is to get sufficient market saturation and dependency of their clients and if they achieve that, they will hike up the prices quite a bit.

(Of course this all depends on if their plans work out, but if not then the smaller local models will probably play a big role in the failure of their plan)

Re:IBM CEO says open weight models are now irresistibly good... by murderfs on Monday October 26, 2026 @05:05AM (#50241054)
People keep repeating this, but it is only remotely close to true for the literal free plans and the small subset of consumers maxing out the pro plans.

At API pricing (i.e. what all enterprise customers are paying), inference is profitable, even if they're paying usurious rates on cost of capital for capex. Where the frontier labs are getting screwed is that they're in a Red Queen's race with each other, spending tens of billions of dollars per year training the next frontier model, and not being able to recoup the investment on that model before it's obsolete.

That's why you see the CEOs calling for government intervention to "pace the frontier": it lets them reduce investment on training without risking that their competitors keep going and make them obsolete. Unfortunately for them, the Chinese labs exist, and would love nothing more than to take the frontier, so they're in a bit of a pickle.

Re:IBM CEO says open weight models are now irresistibly good... by roca on Monday October 26, 2026 @05:28AM (#50241227)
The people I personally know who signed the "pacing the frontier" petition (including me), or who called for it in other ways, are sincere in their concerns that things are moving too fast for safety.
Re:IBM CEO says open weight models are now irresistibly good... by ThunderSizzle on Monday October 26, 2026 @06:30AM (#50241651)
Put the liability on the AI companies, and they'll slow down.

However, that also means AI will be even more lobotomized since the cloud providers will panic over being liable

Re:IBM CEO says open weight models are now irresistibly good... by root_axis on Monday October 26, 2026 @05:29AM (#50241231)
> inference is profitable

People keep repeating this, but it is only remotely close to true if you evaluate inference in a vacuum without any of the other supporting costs to facilitate that capability.

Just a few years ago people were saying the same thing about WeWork: "leasing is profitable, the unit economics of individual locations is net positive", and yet...

Re:IBM CEO says open weight models are now irresistibly good... by cgio on Monday October 26, 2026 @04:39AM (#50240864)
Do you take into account the token allowance? For a heavy user this would be the equivalent of many subscriptions and they add up.
Re:IBM CEO says open weight models are now irresistibly good... by m4rtink on Monday October 26, 2026 @06:18AM (#50241563)
But what if you become unable to pay for it ? What if:

- they raise the prices 10x

- go bankrupt

- ban you or your country

- decide not to help you with what you want to do for opaque reasons

- you are in jail because the model halucinated something and the company promptly reported you to the police

Better avoid all off these and go local IMHO.

Re:IBM CEO says open weight models are now irresistibly good... by pixl97 on Monday October 26, 2026 @10:32AM (#50244413)
>you are in jail because the model halucinated something and the company promptly reported you to the police

Just note that your local model can 100% do this itself.

Re:IBM CEO says open weight models are now irresistibly good... by windexh8er on Monday October 26, 2026 @07:23AM (#50241972)
This trope minimizes so many downsides of "frontier" models. When you can't do what you need to with a model it really isn't valuable - which is why subscriptions still exist. As soon as you're forced to pay actual token costs people who keep saying this will likely wish they had invested in something else earlier.

It's also very clear that providers are running variants of models, especially at the SotA levels. I wouldn't be surprised to find out at some point that 1-2 months after release OAI and Anthropic just run "good enough" quants to save themselves on inference (since they're losing money hand over fist). Another upside of local models is that I don't need to worry about model degradation at any given point.