News for hackers. Stuff that matters.
Re:A few words on DS4 by gcr on Thursday May 14, 2026 @09:50PM (#48143570)
DwarfStar4 is a small LLM inference runtime that can run DeepSeek 4. The blog post implies that it currently requires 96GB of VRAM.

For others who are lacking context :-)

Re:A few words on DS4 by foresto on Thursday May 14, 2026 @09:55PM (#48143616)
Thanks. Outside of LLM circles, DS4 is usually a video game controller.
Re:A few words on DS4 by artyom on Thursday May 14, 2026 @10:23PM (#48143799)
Well, I was sitting here expecting the Redis creator have an opinion on still-unannounced Dark Souls 4.
Re:A few words on DS4 by low_tech_love on Friday May 15, 2026 @03:38AM (#48145670)
Haha the same here!!
Re:A few words on DS4 by insensible on Friday May 15, 2026 @12:07AM (#48144450)
Trekkies are experiencing a major regression from Deep Space Nine.
Re:A few words on DS4 by RALaBarge on Friday May 15, 2026 @06:21AM (#48146817)
They never should have trusted Qwark
Re:A few words on DS4 by kjs3 on Friday May 15, 2026 @10:06AM (#48148782)
There were prototypes. The Cardassians never get it right the first (eight) times.
Re:A few words on DS4 by burnte on Friday May 15, 2026 @03:38PM (#48152889)
Deep Space 4 vanished and was never seen again.
Re:A few words on DS4 by oezi on Friday May 15, 2026 @12:04AM (#48144429)
Or a car from Citroen
Re:A few words on DS4 by pavlov on Friday May 15, 2026 @02:48AM (#48145335)
Technically DS is an independent sibling of Citroën within Stellantis, a sprawling car conglomerate that owns a dog’s dinner of car brands in Europe and USA.
Re:A few words on DS4 by orthoxerox on Friday May 15, 2026 @04:57AM (#48146214)
It's still the Lexus to Citroen's Toyota.
Re:A few words on DS4 by drcongo on Friday May 15, 2026 @04:59AM (#48146229)
Pavlov's dog's dinner?
Re:A few words on DS4 by Hamuko on Friday May 15, 2026 @05:44AM (#48146558)
If we want to get really technical, “DS4” is a model from Citroën and they later spun out the DS lineup into its separate brand, with the “Citroën DS4” becoming “DS 4”, “DS” being the make and “4” being the model.
Re:A few words on DS4 by pavlov on Friday May 15, 2026 @06:08AM (#48146740)
And even more pedantically, DS has recently adopted a new naming scheme where the former DS 4 is now written as DS N°4, pronounced "number 4"...

Their stated inspiration for this SEO bomb is Chanel perfumes.

Re:A few words on DS4 by jofzar on Thursday May 14, 2026 @10:35PM (#48143878)
I am actually kind of disappointed it wasn't a deep dive on the dual shock 4
Re:A few words on DS4 by DeathArrow on Friday May 15, 2026 @12:21AM (#48144522)
>The blog post implies that it currently requires 96GB of VRAM.

From the Github page it seems it only supports Apple and DGX Spark. I have 128 GB of RAM and a 3090 but it probably won't work.

Re:A few words on DS4 by thomasm6m6 on Friday May 15, 2026 @01:11AM (#48144794)
FYI, llama.cpp (which antirez/ds4 is inspired by) supports system ram. E.g. [1] is a good guide for running a similar-sized model with 128gb ram and a 3090-sized GPU.

[1] https://unsloth.ai/docs/models/tutorials/minimax-m27

(Unsloth's deepseek-v4 support is still WIP)

Re:A few words on DS4 by DeathArrow on Friday May 15, 2026 @02:09AM (#48145114)
Thanks, I can run Qwen 3.6 27B with vllm, but I was curious about antirez tool.
Re:A few words on DS4 by embedding-shape on Friday May 15, 2026 @07:48AM (#48147425)
Have you had it getting stuck in endless loops maybe ~10-20% of the invocations? Seems it happens for both the responses and chatcompletion APIs, and no matter what inference parameters I try it happens at least for 1/10 of the requests, I've tried every compatible vLLM version + currently using it from git (#main) yet the issue persists.

Seems to happen with various quantizations too, even the NVFP4 versions and any others, so seems like a deeper issue to me, or hardware incompatible perhaps.

Re:A few words on DS4 by modmans2nd on Friday May 29, 2026 @07:21AM (#48321711)
There’s a fixed version out there with corrected templates.
Re:A few words on DS4 by manmal on Friday May 15, 2026 @01:59AM (#48145062)
It wouldn’t be useful with your setup, probably 3-4 token per second.
Re:A few words on DS4 by DeathArrow on Friday May 15, 2026 @03:07AM (#48145458)
Yep, maybe I can open a feature request if it makes sense technically.
Re:A few words on DS4 by zozbot234 on Friday May 15, 2026 @04:05AM (#48145834)
Arguably it makes more sense technically to get the model support into llama.cpp, which provides many options for GPU+CPU split inference already.
Re:A few words on DS4 by hellifino on Saturday May 16, 2026 @11:20AM (#48160987)
I have an AMD 3995wx and 128GB DDR4 3200 I can load the Q2 and using -t 64 can get around 4 t/s out of the box. Havent tried any other configs yet.

I do not think it can use multi-gpu or gpu/cpu offloading at this time.

Re:A few words on DS4 by zozbot234 on Saturday May 16, 2026 @12:25PM (#48161581)
That sounds memory bandwidth limited. Does the total t/s decode throughput improve by running multiple sessions in parallel?

(Note, that's total not per-session. Tok/s figures per session will initially tank since you're using the same total mem bandwidth to load incrementally more active params.)

Re:A few words on DS4 by zozbot234 on Friday May 15, 2026 @02:49AM (#48145336)
> The blog post implies that it currently requires 96GB of VRAM.

Has anyone tested what happens if you try and run this on lower-RAM Macs? It might work and just be a bit slower as it falls back on fetching model layers from storage.

Re:A few words on DS4 by conradkay on Friday May 15, 2026 @03:03AM (#48145414)
It'd be way slower since you'd be doing that work every token
Re:A few words on DS4 by zozbot234 on Friday May 15, 2026 @03:08AM (#48145470)
True (with 64GB RAM it'd have to fetch 20% of its active experts from disk already, about 650MB/tok at 2-bit quant - and that percentage rises quickly as you lower RAM further); my question is just a more practical one about whether it runs at all, how bad the slowdown is, and to what extent you might be able to get some of that decode throughput back by running multiple (slower) agent sessions in parallel under a single Dwarf Star 4 server.
Re:A few words on DS4 by computably on Friday May 15, 2026 @03:43PM (#48152928)
Storage is multiple orders of magnitude slower than RAM. Pretty sure it'd be more like 10s/tok than anything reasonable.
Re:A few words on DS4 by zozbot234 on Friday May 15, 2026 @04:01PM (#48153121)
Active params for this model is 13B which takes about 6.5GB at full native quantization, or perhaps 3.25GB at the 2bit quant that's being provided here, that should take significantly less than 10s to fetch on Mac storage, especially given that some fraction of the model weights would be cached in RAM. Sounds like something worth testing out if it can be made to work out of the box with DS4.
Re:A few words on DS4 by rpigab on Friday May 15, 2026 @04:05AM (#48145836)
I knew Death Stranding 3 wasn't out yet!
Re:A few words on DS4 by smcleod on Friday May 15, 2026 @05:41AM (#48146538)
That's the flash version not the full model and only at Q2-3~ so while impressive it's still quite different from the full model.
Re:A few words on DS4 by rurban on Friday May 15, 2026 @06:15AM (#48146773)
Not really. I'm building now another fast C compiler with DeepSeek 4 Flash, and rarely have to step outside to use Pro or Sonnet, gpt or kimi-2.6. Flash is very capable of almost everything.
Re:A few words on DS4 by gekoxyz on Friday May 15, 2026 @11:40AM (#48150050)
which harness are you using? pi? opencode?
Re:A few words on DS4 by rurban on Friday May 15, 2026 @12:21PM (#48150504)
That's not a harness. That's an agent cli. A harness is something completely different. Wish people could use proper terminology.

A test harness is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It automates the execution of test suites, providing the necessary stubs, drivers, and runtime environments so developers can isolate and verify specific code components.

I use opencode (lockedcode is still vaporware), claude, kimi and codex.

And most models. Just no Google models so far, I don't trust them.

Re:A few words on DS4 by Sinidir on Friday May 15, 2026 @02:16PM (#48151925)
Harness: a piece of equipment with straps and belts, used to control or hold in place a person, animal, or object.

So yes the generel meaning applies to test setup and running and also to the agent cli which is the harness for the model.

Re:A few words on DS4 by rurban on Friday May 15, 2026 @04:20PM (#48153312)
No, an agent cli is no harness. You have to provide a harness for an agent by yourself, otherwise it will run free. Which is called vibe coding. Free as you wish, without any harness.
Re:A few words on DS4 by teknologist on Friday May 15, 2026 @05:28PM (#48154131)
An agent cli provides a sandbox, with permission systems and auto command classifiers. That’s part of the harness.
Re:A few words on DS4 by modmans2nd on Friday May 29, 2026 @07:19AM (#48321704)
lol pi doesn’t
Re:A few words on DS4 by computably on Friday May 15, 2026 @03:02PM (#48152495)
Akshually, they said "harness," and not "test harness."

There's no particular reason "agent harness" can't have practically the same definition, substituting test-specific concepts for agent-specific ones.

Re:A few words on DS4 by tredre3 on Friday May 15, 2026 @05:06PM (#48153848)
You're free to fight the terminology if you want (I did at first too), but the zeitgeist has chosen a meaning that disagrees with you, so people will see you as being deliberately obtuse and unpleasant when you fight back.

Learning when to let go is an incredibly important skill that I have learned way too late in life.

Re:A few words on DS4 by dolmen on Friday May 15, 2026 @06:50PM (#48154956)
May I ask about your trust issue regarding Google models?

Is it about quality issues (lack of guardrails, agent runs dangerous commands)? I have seen first-hand Gemini-cli going out of the project directory and using my home directory as a work area.

Or is it about terms of service?

Or other concerns?

Re:A few words on DS4 by rurban on Saturday May 16, 2026 @01:37AM (#48157133)
Quality. They are too dumb.

And the lack of ease of use.

Re:A few words on DS4 by modmans2nd on Friday May 29, 2026 @07:19AM (#48321697)
It is really telling when people say that. It’s clear they think the job of harness development is done by selecting the agent environment.
Re:A few words on DS4 by Wowfunhappy on Friday May 15, 2026 @06:40AM (#48146952)
Thanks. How is DwarfStar4 different from llama.cpp?
Re:A few words on DS4 by covoeus on Friday May 15, 2026 @04:02PM (#48153129)
llama.cpp is general purpose in the sense that it supports many different model architectures. ds4 is laser focused on deepseek v4 flash, thus having a leaner codebase