Re:Nvidia DGX Spark encrypted firmware slows inference speed...
by KeplerBoy on Wednesday November 04, 2026 @06:05AM
(#50363887)
The RTX Spark is also sold in a mac mini form factor. That's pretty well suited for home inference.
[ Parent
| on HN ]
Re:Nvidia DGX Spark encrypted firmware slows inference speed...
by barrkel on Wednesday November 04, 2026 @08:18AM
(#50365148)
Exactly. Get yourself a DGX spark and plug it in at home.
I meanwhile, run my ridiculously OP workstation class hardware as a server - 9995wx, 768GiB, rtx pro 6000 w/ 96GiB vram - at home, and my Asus ZenBook is light, cool, and long-lived.
Runs GLM 5.3 flash at 35 tok/sec while consuming only 15 GiB VRAM, and I can run a lot more stuff on there alongside.
Re:Nvidia DGX Spark encrypted firmware slows inference speed...
by jacquesm on Wednesday November 04, 2026 @10:30AM
(#50367002)
That's slower than I would have expected for the RTX 6000, do you have figures for say Qwen 3.8 27B? That model fits completely on the card, that should push it a lot further.
Re:Nvidia DGX Spark encrypted firmware slows inference speed...
by barrkel on Thursday November 05, 2026 @08:47AM
(#50379014)
I'm pretty certain I can get it to go faster by caching experts and whatnot (with experimental llama.cpp branches), but that would consume a lot more VRAM and it's fast enough as it is for playing around with, and I can leave it always loaded. It's like 50% CPU inference, with a few layers + router + kv cache on the GPU.