News for hackers. Stuff that matters.
Re:Ask HN: People who switched from GPT to their own models.... by gettodachoppa on Tuesday February 27, 2024 @09:01AM (#39524184)
https://docs.sillytavern.app/usage/local-llm-guide/how-to-us...

Follow the guide all the way until you get to "Loading our model in Oobabooga". Then ignore the rest. You can do inference in Ooba under the Notebook tab.

(You can also ignore the "enabling HTTP API" parts, but it's quite handy, it's an OpenAI-compatible API which means you can use any OpenAI-compatible web UI)

There are no comments on this comment yet.