News for hackers. Stuff that matters.

A comment on High-Throughput Generative Inference of Large Language Models with a Single GPU. Parent comment

Re:High-Throughput Generative Inference of Large Language Mo... by hooloovoo_zoo on Monday March 13, 2023 @11:18PM (#35146965)
Bear in mind that 1 token/s is useful for like… correspondence chess as far as games are concerned.
[ Parent | on HN ]

There are no comments on this comment yet.