Re:LLaMa running at 5 tokens/second on a Pixel 6
by Havoc on Wednesday March 15, 2023 @01:06PM
(#35171371)
Any more details? I'm guessing they're leveraging the NPU in the pixel?
[ Parent
| on HN ]
Re:LLaMa running at 5 tokens/second on a Pixel 6
by saidinesh5 on Wednesday March 15, 2023 @01:23PM
(#35171703)
I think they are using llama.cpp without any NPU/TPU patches. By default it only runs on CPU with support for various SIMD extensions.