News for hackers. Stuff that matters.
Re:70B Llama 2 at 35tokens/second on 4090 by antupis on Wednesday September 13, 2023 @08:48AM (#37496042)
Yup, but the thing is that can you get model working how you want and can you run it production. My experience is that gpt4 yes, gpt3.5 yes but needs some work Llama2-70b 8-bit or more yes but needs some work after that it really depends usecase.

There are no comments on this comment yet.