Coding with Llama 3.1, New DeepSeek Coder and Mistral Large

Re:Coding with Llama 3.1, New DeepSeek Coder and Mistral Large by anotherpaulg on Thursday July 25, 2024 @05:14AM (#41066526)

Five noteworthy models have been released in the last few days, with a wide range of code editing capabilities. Here are their results from aider’s code editing leaderboard with Sonnet and GPT-3.5 included for scale.

  77% claude-3.5-sonnet
  73% DeepSeek Coder V2 0724
  66% llama-3.1-405b-instruct
  60% Mistral Large 2 (2407)
  59% llama-3.1-70b-instruct
  58% gpt-3.5-turbo-0301
  38% llama-3.1-8b-instruct

[ Parent | on HN ]

Re:Coding with Llama 3.1, New DeepSeek Coder and Mistral Large by WiSaGaN on Thursday July 25, 2024 @10:29AM (#41069095)

Exciting! So many models dropped on the 24th. Really to know whether the numerical suffix means YYMM or MMDD!

[ Parent | on HN ]

Re:Coding with Llama 3.1, New DeepSeek Coder and Mistral Large by tarasglek on Thursday July 25, 2024 @10:57AM (#41069457)

Been experimenting with aider for a while. Really fun using it for basically free with deepseek. Thanks for getting that to work.

My question is re your whole file fallback approach. Since you already have everything mapped out with treesitter, have you considered doing function at a time instead? It's just as easy for a model to ask for a function as it is for a file

[ Parent | on HN ]