Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Inkling 2 Mini. Parent comment
Looking a bit oof on reddit rn, I'm seeing reports of 10 tok/s via llama.cpp :|
https://www.reddit.com/r/LocalLLaMA/comments/1y1g9cr/inkling...
Edit: Hopeful that this gets the ninfer treatment, if the benchmarks are to be believed it feels like there is probably both amazing speed and real performance to be unlocked by the community here on the x090 cards
There are no comments on this comment yet.
Looking a bit oof on reddit rn, I'm seeing reports of 10 tok/s via llama.cpp :|
https://www.reddit.com/r/LocalLLaMA/comments/1y1g9cr/inkling...
Edit: Hopeful that this gets the ninfer treatment, if the benchmarks are to be believed it feels like there is probably both amazing speed and real performance to be unlocked by the community here on the x090 cards