News for hackers. Stuff that matters.

A comment on Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa. Parent comment

Re:Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generat... by londons_explore on Wednesday March 29, 2023 @05:41AM (#35354755)
That is likely a bug. There is nothing in the maths that should make prompt tokens slower on CPU.
[ Parent | on HN ]
Re:Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generat... by loufe on Wednesday March 29, 2023 @09:48AM (#35357242)
It is a bug, it's been clearly discussed under the issues of LLaMA.cpp. The wait time is far from atrocious, with a 5600x and the 13B I would wait 15 seconds before input starts for a relatively complex prompt.
[ Parent | on HN ]