inLewOf

Not too late to change the logo to a friendly piece of halloumi :) (best i could find from a quick google image search https://www.redbubble.com/i/poster/Halloumi-by-PaulSDesign/1...)

re Temperature config option: I've found it useful for trying to generate something akin to a sampling-based confidence score for chat completions (e.g., set the temperature a bit high, run the model a few times and calculate the distribution of responses). Otherwise haven't figured out a good way to get confidence scores in llama.cpp (Been tracking this git request to get log_probs https://github.com/ggerganov/llama.cpp/issues/6423)