News for hackers. Stuff that matters.
Re:LLaMA Now Goes Faster on CPUs by jrflowers on Monday April 01, 2024 @02:43AM (#39891455)
> If you want factual answers from a local model it might help to turn the temperature down.

This makes sense. If you interact with a language model and it says something wrong it is your fault

Re:LLaMA Now Goes Faster on CPUs by astrange on Monday April 01, 2024 @04:52AM (#39892011)
You're not "interacting with a language model", you're running a program (llama.cpp) with a sampling algorithm which is not set to maximum factualness by default.

It's like how you have to set x264 to the anime tuning or the film tuning depending on what you run it on.