Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on ChatGPT's 1-Year Anniversary: Are Open-Source Large Language Models Catching Up?. Parent comment
Tried turning on mlock and upping thread count to 6, but it's still rather slow at around 3 tokens / sec.
There are no comments on this comment yet.
Tried turning on mlock and upping thread count to 6, but it's still rather slow at around 3 tokens / sec.