OpenAI Models Dominate Structured Code Edit Benchmark

Posted by biobootloader on Tuesday January 16, 2024 @02:50PM
from the blog.mentat.ai dept.

( 1 comment | 4 points | on HN )

Re:OpenAI Models Dominate Structured Code Edit Benchmark by granawkins on Tuesday January 16, 2024 @05:52PM (#39020279)

I saw the same thing with Mailogy.

I only tested variants of gpt-3.5 and -4 but got ~50% invalid syntax errors with 3.5, and virtually none with 4.

[ Parent | on HN ]