Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on GPU-Accelerated LLM on a $100 Orange Pi. Parent comment
Otherwise ~1.5 tokens/s is definitely the minimum you'd want streaming tokens to a single person.
There are no comments on this comment yet.
Otherwise ~1.5 tokens/s is definitely the minimum you'd want streaming tokens to a single person.