Show HN: Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

Re:Show HN: Beating GPT-4 on HumanEval with a fine-tuned Cod... by syntaxing on Friday August 25, 2023 @09:01PM (#37268922)

I used textgen, instruct mode with LLaMa 2 template, “simple” parameters. What quantization method are you using?

[ Parent | on HN ]

Re:Show HN: Beating GPT-4 on HumanEval with a fine-tuned Cod... by gmm1990 on Friday August 25, 2023 @10:09PM (#37269296)

I used the llama.cpp quantize q_0 option. Seemed to vary widely just adjusting the temperature parameters. Was giving it an example db query saving to file in json and asking create a similar function did multiple queries.

[ Parent | on HN ]

Re:Show HN: Beating GPT-4 on HumanEval with a fine-tuned Cod... by syntaxing on Friday August 25, 2023 @10:41PM (#37269472)

I’m assuming typo and you mean 4? What model size? As mentioned above, I had serious issues making GGUF work well. I’m gonna post an article talking about a better way to set it up.

[ Parent | on HN ]