We fine-tuned Llama and got 4.2x Sonnet 3.5 accuracy for code generation

Re:We fine-tuned Llama and got 4.2x Sonnet 3.5 accuracy for ... by aussieguy1234 on Thursday January 02, 2025 @07:28AM (#42573846)

You make a bold marketing claim, 4.2x Sonnet, but viewing your website, I can see no data or test results to back this up.

[ Parent | on HN ]

Re:We fine-tuned Llama and got 4.2x Sonnet 3.5 accuracy for ... by mathgeek on Thursday January 02, 2025 @07:42AM (#42573939)

Thanks for calling this out, even if it just gets OP to comment with some details/data. Was hoping this would be a shallow or deep dive into the results, but looks like it’s just a marketing post to a marketing page to support a PH launch.

[ Parent | on HN ]

Re:We fine-tuned Llama and got 4.2x Sonnet 3.5 accuracy for ... by samatdav on Thursday January 02, 2025 @11:58PM (#42582598)

Good point, I agree, we haven't shared enough details. Since we are very early, we only got high level results and want to get feedback on what direction would be most applicable and useful. We plan to add more metrics and data to the website in the future and also want to publicly host a fine-tuned model for anyone to try and see.

[ Parent | on HN ]