ttsiodras

You read that right. From the article:

"Affected products: ... Up to (excluding) 152.0.7977.82"

.82 is fixed.

Just chiming in - the claims above are real, I have very similar numbers in a cluster of 2x GX10 I have access to.

Instructions to reproduce, and benchmarks here: https://forums.developer.nvidia.com/t/deepseek-v4-flash-offi...

I hear you on the insane amount of time vllm takes to launch (atlas is a move in the right direction in that regard).

But mostly I wanted to raise awareness to readers of your article that no, if you want to do inference, paying 15K for a single 96GB card almost certainly makes no sense. Buy 4 GX10s with the same money, and enjoy dramatically better models and user scalability.

Regardless - thanks for putting the effort to share your findings! I keep postponing doing the same... there's tons of things everyone is re-discovering on their own.

Interesting article.

IMHO, the author could have done two things better:

- vllm instead of llama.cpp. With NVIDIA HW, there is huge difference in multi-user loads and caching with vllm; when he was complaining about what happens when more than one user uses the model, and about losing caching, I was "well, duh".

- The budget he used for a single card could have instead be put to far, far better use with SPARKs. I have access to a cluster of 2 x GX10 - total cost less than half what he paid, even today - and I am running vllm and Deepseek v4 Flash. The difference compared to any Qwen is tremendous - I've NEVER seen it loop, and in all my experiments so far, it's the most Sonnet-y model I've ever tried (antirez seems to agree, hence his ds4 fork).

If you're wondering about how I set it up in the 2 GX10s: https://forums.developer.nvidia.com/t/deepseek-v4-flash-offi...

Performance: 2K t/s prefill ( very useful for feeding tons of source code into its massive context window ) and around 50-60 tg/s in my coding sessions in the pi.dev harness. With the money the author paid, he could have bought 4 GX10s, and double both numbers ( vllm basically scales almost linearly with tensor parallelism ).

Thanks, Mark - for both setups!

It's very nice seeing it put to use in actual Spectrum machines - love it :-)

Oh wow! I never expected to see this in a real Speccy until I fix the one I have back home (I'm an expat).

Thank you so much! :-)

Thank you!

I have been using xterm's default dabbrev-expand to do the same via "Alt-/" ( https://github.com/ttsiodras/dotfiles/blob/master/.Xresource... ) - which works regardless of what shell you're in.

But I was curious for your approach... so I asked Claude to convert it to bash: https://claude.ai/public/artifacts/01a49347-1617-4afe-8476-0...

Works like a charm - pinned it to Ctrl-k, which was free in my setup. I guess I don't have to depend on XTerm for this any more :-)

Thanks!

As soon as I met Forth, I felt I had to hack my own (https://github.com/ttsiodras/MiniForth/) - and did so over a single week, two years ago. I targeted the Blue Pill and the original Arduino, but developed via cross-compilers so that I could test my code in the host.

I became so obsessed with the project that I was looking forward to tinkering with it after coming back from work every day; so it was hacked in 5 evenings and a weekend. It was that much fun, to build a Forth.

I highly recommend the process; I think the only other time I felt so enlightened was when I first met Lisp macros (https://www.thanassis.space/score4.html#lisp).

This is why I come to HN - brilliant ideas that I should have thought of before, but didn't!

I just changed my server, and uninstalled the - now truly useless - fail2ban. I use SSH keys of course, but without fail2ban my server's logs were constantly flooded with hacking attempts.

No longer - wireguard for the win. Thank you, chlorion!

signal-to-noise ratio: off the charts.

Many thanks, OP.

> dkozel and your kind, whoever you are, wherever you are, and whatever you are doing right now, you are legend.

Both of you are :-)

Heart=warmed. Happy holidays to both of you!

Thanks for sharing! The slides are here: http://vger.kernel.org/~acme/prez/linux-plumbers-2022/

I was not aware of gsocket - thank you, very useful!