Re:Running LLaMA 7B on a 64GB M2 MacBook Pro with Llama.cpp
by lxe on Sunday March 12, 2023 @08:09PM
(#35128606)
How are you getting 20 tokens/second? I'm getting 2.6 tokens/s on 3090 with int4 prequantized model. Is 4090 so much faster?
[ Parent
| on HN ]
There are no comments on this comment yet.