Re:Dalai: Automatically install, run, and play with LLaMA on...
by terafo on Sunday March 12, 2023 @08:22PM
(#35128768)
It fits, whisper.cpp uses 4 bit quantization, 13B model takes a little bit more than 8gb and around 9gb ram while inferencing.
[ Parent
| on HN ]
There are no comments on this comment yet.