News for hackers. Stuff that matters.

A comment on Refact LLM: New 1.6B code model reaches 32% HumanEval and is SOTA for the size.

Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by acheong08 on Monday September 04, 2023 @02:13PM (#37383246)
Say I want to fine tune a Golang specific model. How much $ and effort would I have to put in? Would using this as a base help in any way compared to starting from llama?
[ Parent | on HN ]
Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by OlegKlimov1337 on Monday September 04, 2023 @02:25PM (#37383355)
Maybe it makes sense to start from llama-code, not llama :D I think golang specific model will not be that different from a multi-language model. But it definitely will work better after fine tuning on your code. Check out refact self hosting docker in a couple of days, finetune will be there soon. It will take you 1 GPU and almost no money )
[ Parent | on HN ]