News for hackers. Stuff that matters.

A comment on The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT. Parent comment

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by alxfoster on Monday April 10, 2023 @10:32AM (#35512795)
FYI, many of us are indeed running 65B. I’m running 65B at 4-bit and getting about 7.5 tokens per second. Granted, I have a beefy machine with 2x 3090s and Nvlink but certainly well within the realm of any small lab.
[ Parent | on HN ]

There are no comments on this comment yet.