News for hackers. Stuff that matters.
Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by JegernOUTT on Tuesday September 05, 2023 @07:29AM (#37390314)
It can be checked if the model predicts canonical solutions from humaneval. I understand it is not ideal, but at least you can check it yourself

There are a bunch of other benchmarks too, check out the page https://huggingface.co/smallcloudai/Refact-1_6B-fim

Also, feel free to run any new benchmarks

There are no comments on this comment yet.