News for hackers. Stuff that matters.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by impossiblefork on Wednesday November 04, 2026 @05:30AM (#50363614)
Unrelated to this slowdown, I could imagine this sort of thing being the right application for things like Taalas, although I'm not quite sure whether it can actually offer what I imagine that it can (i.e. ability to make a cheap, physically small card that can run a large model).

A laptop can't run the latest big models, because memory is too expensive. This thing probably can't run models larger than Mistral Small 4. But it'd be a completely different thing to have a hardcoded Mistral Large 4, than a choice of tiny models.

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by RealityVoid on Wednesday November 04, 2026 @05:43AM (#50363718)
You probably won't be able to run a sota model on a single Talaas card. You probably need a set. It should theoretically be cheaper, faster, more power efficient than Nvidia at running that model. But then you are stuck with a chip that can run only that model. And I understand that context size will also be a problem.
Re:Nvidia DGX Spark encrypted firmware slows inference speed... by impossiblefork on Wednesday November 04, 2026 @05:54AM (#50363796)
Yeah, that was why I felt that, while Taalas might not be the right thing in general, it could be really appealing in a laptop, that would otherwise not be able to run any big model at all.

I think I misunderstood what the technology could do a little bit, because I imagined these fixed matrix multiplies encoded into hardware as small-ish, as using routing space etc., not just fast.

I think what I really imagined was useful would be more easily achieved by having a big ROM with model weights connected to an ordinary GPU.

Re:Nvidia DGX Spark encrypted firmware slows inference speed... by wmf on Wednesday November 04, 2026 @12:42PM (#50368957)
Taalas won't be cheap. Each chip will be at least $1,000 and large models require multiple chips (maybe hundreds).