News for hackers. Stuff that matters.

A comment on 70B Llama 2 at 35tokens/second on 4090. Parent comment

Re:70B Llama 2 at 35tokens/second on 4090 by Havoc on Wednesday September 13, 2023 @02:39PM (#37500060)
I doubt a 2.55bit quantised 70bn model will beat gpt3.5 on a meaningful percentage of tasks
[ Parent | on HN ]

There are no comments on this comment yet.