News for hackers. Stuff that matters.
Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by ALittleLight on Monday September 04, 2023 @02:26PM (#37383365)
How does it compare to Copilot? A metric I'd like to see is % of proposed completions accepted by a human user. If you had an extension that 50% of the time proposed a Copilot extension and 50% of the time proposed a Refact extension (blind to the user) then you could come up with a metric like this.

There are no comments on this comment yet.