News for hackers. Stuff that matters.
Re:Google “We Have No Moat, and Neither Does OpenAI” by kiratp on Thursday May 04, 2023 @10:13PM (#35824535)
Benchmarks for what you can do on CPU alone.

https://github.com/ggerganov/llama.cpp/issues/34

An M1 Max does 100ms per token. A 64 core threadripper about 33ms per token.

There are no comments on this comment yet.