Re:How Is LLaMa.cpp Possible?
by moffkalast on Tuesday August 15, 2023 @07:56PM
(#37140951)
The problem is that it's not consistent enough for a good demo. Not even two different models, but even two different fine tunes of the same base model may be wildly differently affected by quantization. It can range from making hardly a difference to complete garbage output.
[ Parent
| on HN ]
There are no comments on this comment yet.