We fine-tuned Llama and got 4.2x Sonnet 3.5 accuracy for code generation

Re:We fine-tuned Llama and got 4.2x Sonnet 3.5 accuracy for ... by WhatsName on Thursday January 02, 2025 @06:51AM (#42573665)

Talk is cheap, benchmarks please. Also why did you decide for LLama? AFAIK deepseek always had a slight edge over llama when it comes to coding performance, or is this no longer the case?

[ Parent | on HN ]

Re:We fine-tuned Llama and got 4.2x Sonnet 3.5 accuracy for ... by KTibow on Thursday January 02, 2025 @07:41AM (#42573935)

Training and running fine tunes of Deepseek could get expensive.

[ Parent | on HN ]

Re:We fine-tuned Llama and got 4.2x Sonnet 3.5 accuracy for ... by samatdav on Thursday January 02, 2025 @11:48PM (#42582551)

Good point, we plan to publish more benchmarks and also publicly host a model for anyone to try. We think Llama is a good option but as we progress we will test other open source models too like deepseek.

[ Parent | on HN ]