News for hackers. Stuff that matters.
Re:RasaGPT: First headless LLM chatbot built on top of Rasa,... by MacsHeadroom on Monday May 08, 2023 @10:33PM (#35869571)
>compared to the 1token/second claimed by the p40 user

That user is doing something wrong. They may not be cooling it and are getting thermal throttled. That would be my guess.

The P40 is capable of upwards of 10 tokens/second with 30b.

There are no comments on this comment yet.