News for hackers. Stuff that matters.

A comment on The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT. Parent comment

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by AeiumNE on Sunday April 09, 2023 @04:41PM (#35506283)
It's possible to run with a cpu and then use system memory.
[ Parent | on HN ]
Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by alfor on Sunday April 09, 2023 @08:32PM (#35507857)
Do you know what kind of speed you can expect with inference on CPU?
[ Parent | on HN ]
Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by MacsHeadroom on Monday April 10, 2023 @02:19AM (#35509671)
Yes, around 2 tokens per second on a newer CPU with fast RAM.
[ Parent | on HN ]