We used no prompt (only autocomplete) for the HumanEval reproduction: https://huggingface.co/Phind/Phind-CodeLlama-34B-v1.

But you should be able to get it to do stuff just by telling it what you want. Note that it's completion tuned (not chat-tuned), so it should perform better on single-turn conversations.

I used textgen, instruct mode with LLaMa 2 template, “simple” parameters. What quantization method are you using?

I used the llama.cpp quantize q_0 option. Seemed to vary widely just adjusting the temperature parameters. Was giving it an example db query saving to file in json and asking create a similar function did multiple queries.

I’m assuming typo and you mean 4? What model size? As mentioned above, I had serious issues making GGUF work well. I’m gonna post an article talking about a better way to set it up.