Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Show HN:Getting GLM 5.2 running on my slow computer.
Do you mmap or issue reads on demand? Also do you use io_uring to interleave compute with io or do you spawn extra threads?
I also tried predicting which experts get reused and I managed to beat a simple LRU very slightly.
EDIT: That was on Kimi 2.5 but even worse quant than 4bit. IIRC it was 2.6 or so
There are no comments on this comment yet.
Do you mmap or issue reads on demand? Also do you use io_uring to interleave compute with io or do you spawn extra threads?
I also tried predicting which experts get reused and I managed to beat a simple LRU very slightly.
EDIT: That was on Kimi 2.5 but even worse quant than 4bit. IIRC it was 2.6 or so