News for hackers. Stuff that matters.
Re:ChatGPT's 1-Year Anniversary: Are Open-Source Large Langu... by gardnr on Friday December 01, 2023 @06:38AM (#38485662)
I can fit 13B Q4 K M models on a 12GB RTX 3060. It OOMs when the context window goes above 3k. I get 25 tok/s.

There are no comments on this comment yet.