News for hackers. Stuff that matters.
Re:Dalai: Automatically install, run, and play with LLaMA on... by terafo on Sunday March 12, 2023 @08:22PM (#35128768)
It fits, whisper.cpp uses 4 bit quantization, 13B model takes a little bit more than 8gb and around 9gb ram while inferencing.

There are no comments on this comment yet.