Refact LLM: New 1.6B code model reaches 32% HumanEval and is SOTA for the size

Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by riku_iki on Monday September 04, 2023 @02:10PM (#37383226)

so, we have to trust your procedure..

[ Parent | on HN ]

Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by JegernOUTT on Tuesday September 05, 2023 @07:29AM (#37390314)

It can be checked if the model predicts canonical solutions from humaneval. I understand it is not ideal, but at least you can check it yourself

There are a bunch of other benchmarks too, check out the page https://huggingface.co/smallcloudai/Refact-1_6B-fim

Also, feel free to run any new benchmarks

[ Parent | on HN ]