News for hackers. Stuff that matters.
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by adeon on Thursday March 16, 2023 @04:14PM (#35187792)
I have not, but I want to in near future because I'm really curious myself too. I've been following Rust community that now has llama.cpp port and also my OpenCL thing and one discussion item has been to run a verification and common benchmark for the implementations. https://github.com/setzer22/llama-rs/issues/4

I've mostly heard that, at least for the larger models, quantization has barely any noticeable effect. Would be nice to witness it myself.

There are no comments on this comment yet.