News for hackers. Stuff that matters.
Re:Anthropic's Safety Superpower by embedding-shape on Tuesday June 16, 2026 @05:03PM (#48562039)
ds4: prefill: 150.18 t/s, generation: 48.47 t/s together with https://github.com/antirez/ds4 + RTX Pro 6000, local provider :) I guess you could rent one too, get similar speed. https://gist.github.com/embedding-shapes/eceefa1b2699dca448f...

The weights I've tried is the "DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix" ones which fits just about within 96GB VRAM. With some tuning, I've managed to get it to get up to ~60 t/s, I'm sure there is other things to do there too :)

There are no comments on this comment yet.