News for hackers. Stuff that matters.

A comment on Refact LLM: New 1.6B code model reaches 32% HumanEval and is SOTA for the size. Parent comment

Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by OlegKlimov1337 on Monday September 04, 2023 @01:35PM (#37382828)
AFAIK There is only one model that do better, it’s phi-1 and it’s python only, and it does not support fill-in-the-middle so you can't really use it.
[ Parent | on HN ]
Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by umutisik on Monday September 04, 2023 @03:32PM (#37384055)
Phi-1-small also scores higher with 350M parameters. It helps to be specific about what the comparison is against when claiming SOTA.
[ Parent | on HN ]