News for hackers. Stuff that matters.
Re:LLaMa running at 5 tokens/second on a Pixel 6 by saidinesh5 on Wednesday March 15, 2023 @01:23PM (#35171703)
I think they are using llama.cpp without any NPU/TPU patches. By default it only runs on CPU with support for various SIMD extensions.

https://github.com/ggerganov/llama.cpp

There are no comments on this comment yet.