News for hackers. Stuff that matters.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by barrkel on Wednesday November 04, 2026 @05:01AM (#50363389)
I remain puzzled why anyone would want to do inference on a laptop if they have a home internet connection and a phone.

If you really want to do DIY inference, plug it in at home and set up tailscale so you can access it on the go.

No huge battery drain or toasty hot laptop required. Lower cost laptop. Lighter laptop. Longer battery life laptop. Win win win win.

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by solyin on Wednesday November 04, 2026 @05:17AM (#50363505)
Not everyone grew up in a situation where home was a stable dependable place. Not everyone has a home to go to. If you have it with you, they can't mess with it when you're not there. If you grew up in a broken home, you would understand.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by Gigachad on Wednesday November 04, 2026 @05:41AM (#50363703)
And those people are buying $10k laptops to run local AI?
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by jeroenhd on Wednesday November 04, 2026 @05:49AM (#50363757)
Those people won't be buying the Spark laptop, but asking "why anyone would want to do inference on a laptop" is broader than what the article covers.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by c7b on Wednesday November 04, 2026 @07:58AM (#50364931)
Local AI is an expensive hobby. You're paying big money for small performance (so mainly for the feeling of "owning it", which is a great motivator). It can't match cloud services, not absolutely nor bang-for-buck, especially if you look a bit for cheaper and freebie options. As someone who loves and promotes local AI, in the situation you described my plea would be to use that money to improve your situation more directly (move out, get support,...). The cost of a laptop that can run somewhat decent models should buy you at least several months of therapy (@1 session/week).
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by throw1234567891 on Wednesday November 04, 2026 @08:42AM (#50365389)
You can't put a price of sending all your IP to a dubious third-party. Sorry. You argument apples vs oranges.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by c7b on Wednesday November 04, 2026 @08:52AM (#50365518)
If you want local AI, which I absolutely encourage you to, I agree with the OP. Get a (mini)-PC and set up a VPN. It's going to save you money compared to a laptop, short- (usually cheaper) and long-term (usually better upgradeability). You'll have more money for therapy, which I would also encourage you to consider if you come from a broken home, as the GP mentioned.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by throw1234567891 on Wednesday November 04, 2026 @08:55AM (#50365563)
Sure, they pay you well at Anthropic or another OpenAI to go and ridicule people?

Maybe you need a therapy. FOMO can be treated, too. Or you're just upset about Trump falling into irrelevance.

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by c7b on Wednesday November 04, 2026 @10:00AM (#50366496)
Maybe you're responding to the wrong thread. The discussion was about laptop vs PC for local AI, which should be pretty equivalent from the perspective of Anthropic, in the context of a broken home. Read the previous chain and calm down.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by throw1234567891 on Wednesday November 04, 2026 @04:20PM (#50371718)
Thanks, sometimes it's good to hear: stop. I'll think about the therapy.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by Gigachad on Wednesday November 04, 2026 @04:32PM (#50371861)
The situation we are talking about here is someone with no fixed home or insecure housing. That kind of person likely isn’t even paying for cloud AI, let alone buying massively overpriced hardware to run it locally.

The market for someone who wants local AI in a laptop for $10k has to be tiny.

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by Danox on Wednesday November 04, 2026 @06:03AM (#50363869)
The spark laptop in its present form is not very good, in fact, it’s disappointing.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by barrkel on Wednesday November 04, 2026 @08:24AM (#50365212)
If the security of your home possessions is an issue, it should be addressed first before you spend thousands on a very stealable laptop that you need to carry everywhere.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by dirasieb on Wednesday November 04, 2026 @10:09AM (#50366658)
homeless RTX Spark owners, that's a new one
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by selimthegrim on Wednesday November 04, 2026 @01:18PM (#50369437)
How many grad students have you met?
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by solyin on Wednesday November 04, 2026 @02:39PM (#50370484)
You grow up, that mentality still stays with you. I know multi-millionaires that do the most broke ass poor people nonsense because they grew up poor.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by pixelpoet on Wednesday November 04, 2026 @05:18AM (#50363509)
I have been saying this for years, and the answer hits a nerve that gets you downvoted into oblivion: people prefer the aesthetics of laptops, simple as that. They will absolutely go to the grave arguing that they need the mobility etc, then just leave it plugged in. They simply prefer how laptops look, and they absolutely hate it when someone dares say it.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by monegator on Wednesday November 04, 2026 @05:43AM (#50363716)
I have a laptop for work. 95% of the time i'd prefer a tower or at least a mini PC with proper cooling, but that remaining 5% i have to be around with the laptop is enough. I don't like the idea of having to remember to sync the projects every time (especially while i'm in the middle of testing something new so it's not like i can just pull the latest commit or sync the latest good version of something)
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by dumberquestions on Wednesday November 04, 2026 @06:19AM (#50364026)
If you do any travel whatsoever a laptop is usually not optional.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by DaSHacka on Wednesday November 04, 2026 @03:08PM (#50370810)
A laptop that is so beefy it can even run GLM 5.3 locally (while sounding like a jet engine, being as bulky as an old-school travelling briefcase, and having a 1.5 hour battery life) though?

GGP's point is just: keep the bulk of your compute at home, where airflow and energy is cheaper (in an opportunity-cost sense). Keep the laptop portable, cool, and light, with long battery life by offloading your own computing tasks to your own hardware.

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by barrkel on Wednesday November 04, 2026 @08:22AM (#50365188)
To be absolutely crystal clear:

I am not saying don't use a laptop.

I am not saying you have to use cloud inference.

I am saying using a laptop for inference is dumb. If you want private inference:

- use a dgx spark or similar plugged in at home

- set up tailscale so you can connect from anywhere

- use your phone as a hotspot on the go

- work on your laptop, using tailscale to connect to your home inference server if you're on the go

This gets you everything except for the few hours you might be on an airplane.

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by pixelpoet on Wednesday November 04, 2026 @08:57AM (#50365574)
100% agreed, and people saying they need a laptop are missing the point; I wasn't saying don't use laptops, either.

The simple point I'm always trying to make is that area vs volume is a fundamental physical difference regarding cooling, which is critical for high performance applications (that's why they're considering an RTX Spark in the first place, right?).

There's so much more I could say about my Strix Halo machine, which fits in the palm of my hand and I have a wonderful portable screen and keyboard+touchpad (and only cost me 2k with 128 GB fast memory, idles at 4w, amazing 32 thread Zen 5 CPU with AVX-512, ...), but it's either preaching to the choir or pissing against the wind.

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by dlahoda on Wednesday November 04, 2026 @05:40AM (#50363689)
private memory.

unique local data and know how not shared with cloud.

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by barrkel on Wednesday November 04, 2026 @08:14AM (#50365101)
What part of what I said implied data was going to the cloud?
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by close04 on Wednesday November 04, 2026 @05:59AM (#50363836)
> why anyone would want to do inference on a laptop

Because reviewers tell them this is is a good use case, and people want to take advantage of the HW they buy [0]. Unfortunately those reviewers didn't take any extra step to validate anything beyond the default config, out of the box.

[0] https://arstechnica.com/reviews/2026/10/the-surface-laptop-u...

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by KeplerBoy on Wednesday November 04, 2026 @06:05AM (#50363887)
The RTX Spark is also sold in a mac mini form factor. That's pretty well suited for home inference.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by barrkel on Wednesday November 04, 2026 @08:18AM (#50365148)
Exactly. Get yourself a DGX spark and plug it in at home.

I meanwhile, run my ridiculously OP workstation class hardware as a server - 9995wx, 768GiB, rtx pro 6000 w/ 96GiB vram - at home, and my Asus ZenBook is light, cool, and long-lived.

Runs GLM 5.3 flash at 35 tok/sec while consuming only 15 GiB VRAM, and I can run a lot more stuff on there alongside.

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by jacquesm on Wednesday November 04, 2026 @10:30AM (#50367002)
That's slower than I would have expected for the RTX 6000, do you have figures for say Qwen 3.8 27B? That model fits completely on the card, that should push it a lot further.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by barrkel on Thursday November 05, 2026 @08:47AM (#50379014)
I'm pretty certain I can get it to go faster by caching experts and whatnot (with experimental llama.cpp branches), but that would consume a lot more VRAM and it's fast enough as it is for playing around with, and I can leave it always loaded. It's like 50% CPU inference, with a few layers + router + kv cache on the GPU.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by jacquesm on Thursday November 05, 2026 @05:37PM (#50385500)
Unused VRAM is a crime now, I heard ;)
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by alerighi on Wednesday November 04, 2026 @06:41AM (#50364252)
Because maybe you don't want to send your private data to a company that could use it for whatever it wants?
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by barrkel on Wednesday November 04, 2026 @08:13AM (#50365095)
Can you explain what data is going where?
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by entrope on Wednesday November 04, 2026 @08:39AM (#50365363)
Your prompts and whatever data they collect from your dev environment go to the inference provider, and typically the customer has neither control nor assurance over what the inference provider (re-)uses that data for.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by Orygin on Wednesday November 04, 2026 @10:52AM (#50367364)
Your prompts and data are going through your own VPN to your own inference hardware. You did not read the post apparently
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by anonym29 on Wednesday November 04, 2026 @08:46AM (#50365445)
GP isn't proposing using closed proprietary models over an API, they're proposing having local hardware serving open weight models at your house/apartment being remotely accessible through an overlay network like Tailscale. The idea being that local inference is very hungry for power and thermal budget, so you run the inference somewhere that power and thermal headroom is abundant (your house/apartment, on a tower, or dedicated unified memory boxes like DGX Spark, Strix Halo, Mac Studio, etc) and then remotely utilize that inference from a regular laptop. You still get all of the benefits of local, private AI inference, without having to expend a laptop's limited power and thermal budget on doing the inference on the laptop itself.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by spaceywilly on Wednesday November 04, 2026 @09:51AM (#50366322)
On the contrary I think in a few years most inference will be done locally. The smaller models have been slowly catching up to frontier for many tasks, and chips are being developed to run large models on phones and laptops (according to Qualcomm CEO they want to run 10T weights on a mobile chip).

To me the simple answer to why is control. If I run a model locally I get to control the model params, the system prompt, all of it. And of course there is no quota. But probably most importantly I am not tied to paying some company a monthly subscription forever.