News for hackers. Stuff that matters.

A comment on 70B Llama 2 at 35tokens/second on 4090. Parent comment

Re:70B Llama 2 at 35tokens/second on 4090 by kilnr on Wednesday September 13, 2023 @06:01AM (#37494630)
I get about the same speed (35 tok/s) out of 13B Llama2 on a 4070 Ti, FWIW.
[ Parent | on HN ]

There are no comments on this comment yet.