Re:GPT-4 Details Leaked
by jsight on Tuesday July 11, 2023 @12:12PM
(#36683083)
I thought this too, but it sounds like the performance of the GPU ultimately holds it back. Maybe it is the software, but I have yet to see a test on Mac silicon that performed well.
[ Parent
| on HN ]
It is not too bad, 7B is doing 432 tokens per second on a Macbook[1].
[1] Video: https://huggingface.co/datasets/huggingface/documentation-im...