Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on GPU-Accelerated LLM on a $100 Orange Pi. Parent comment
You'll get a log_2-based scaling efficiency with nearly any batchsize increase, pending some limitations (memory, etc).
That should be enough at least to roughly sketch it out.
There are no comments on this comment yet.
You'll get a log_2-based scaling efficiency with nearly any batchsize increase, pending some limitations (memory, etc).
That should be enough at least to roughly sketch it out.