Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on RasaGPT: First headless LLM chatbot built on top of Rasa, Langchain and FastAPI. Parent comment
That user is doing something wrong. They may not be cooling it and are getting thermal throttled. That would be my guess.
The P40 is capable of upwards of 10 tokens/second with 30b.
There are no comments on this comment yet.
That user is doing something wrong. They may not be cooling it and are getting thermal throttled. That would be my guess.
The P40 is capable of upwards of 10 tokens/second with 30b.