News for hackers. Stuff that matters.

A comment on RAM Altman: Linux Script Emails Sam Altman Daily How Much Unused RAM in Your PC. Parent comment

Re:RAM Altman: Linux Script Emails Sam Altman Daily How Much... by gobdovan on Sunday November 08, 2026 @05:01PM (#50417827)
Esp. for ML inference he's right, and not just for RAM. I think this should be thought out seriously instead of mocked, or else open models and local inference won't be able to compete with big data centers with exclusive contracts with hyper scalers. There ought to be some sweetspot where one pooled server could serve 50-100 people, with let's say 25 doing concurrent inference and seeing good throughput.

If DeepSeek-sized models will be required in the future to get Opus 5.5 quality, they're not feasible to run for a single person. Pooling usage is necessary to make it efficient.

But even that is pretty painful to sustain efficiently with current hardware prices:

Concurrent requests 25

Aggregate generation 1,000–1,250 tokens/s

Model DeepSeek V4.1 Flash, 552B MoE

Active parameters during decoding 16B

Candidate hardware 4 × H200, 564 GB HBM

Continuous rental at $2.90/GPU-hour $8,352/month

Cost per registered user $84–167/month

[ Parent | on HN ]
Re:RAM Altman: Linux Script Emails Sam Altman Daily How Much... by Brian_K_White on Sunday November 08, 2026 @05:14PM (#50417960)
The efficiency of centralization is not enough excuse to deny individual agency.

The argument for saying that ram is put to better use collected into central hands vs distributed among individuals is the same as for any and all other resources, and it's invalid.

Humans are not ants and should not aspire to the efficiency of ants at the expense of having to be ants.

You get some centralization, and you get some distribution, and it's completely invalid to be jealous of the portion of all resources that is in distribution.

This should indeed be "thought out seriously" and the result is either mockery or torches and daggers. Think that out seriously.

[ Parent | on HN ]
Re:RAM Altman: Linux Script Emails Sam Altman Daily How Much... by usef- on Sunday November 08, 2026 @05:20PM (#50418028)
I think there's a divide here in people who heard the words "it's more efficent than" and are interpreting that statement on its own (it's a true statement), and people who are assuming the statement also implies "we should only give it to data centers" (which I don't think anyone agrees with)
[ Parent | on HN ]
Re:RAM Altman: Linux Script Emails Sam Altman Daily How Much... by gobdovan on Sunday November 08, 2026 @05:40PM (#50418233)
You misinterpreted my comment. I'm advocating for a middle ground where we may pool hardware in small groups for attacking the hyperscaler moat, specifically for ML inference.
[ Parent | on HN ]