Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Qwen3.8-Flash-Next: A New Architecture, Towards Ultimate Cost-Efficiency. Parent comment
On branch of https://github.com/rdaum/eider (for DGX Spark). ~12 tok/sec decode without speculative decoding (will come later)
Still actively working on this. Prefill currently sucks. Will merge to main by end of day.
EDIT: This has now landed on main. Still haven't done MTP speculative decoding boost, but:
80tok/sec prefill, 12 tok/sec decode. ~90GiB or so resident. n-grams paged from disk.
There are no comments on this comment yet.
On branch of https://github.com/rdaum/eider (for DGX Spark). ~12 tok/sec decode without speculative decoding (will come later)
Still actively working on this. Prefill currently sucks. Will merge to main by end of day.
EDIT: This has now landed on main. Still haven't done MTP speculative decoding boost, but:
80tok/sec prefill, 12 tok/sec decode. ~90GiB or so resident. n-grams paged from disk.