News for hackers. Stuff that matters.

A comment on Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook. Parent comment

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by adeon on Thursday March 16, 2023 @04:11PM (#35187740)
I ran it on a 128 RAM machine with a Ryzen 5950X. It's not fast, 4 seconds per token. But it's just about fits without swapping. https://github.com/Noeda/rllama/
[ Parent | on HN ]

There are no comments on this comment yet.