We built a real-world benchmark for AI code review

Re:We built a real-world benchmark for AI code review by steve_avery on Wednesday February 04, 2026 @11:09PM (#46895578)

I'd be interested, but they don't even list any anthropic model on their code review benchmark, so I feel like they haven't really tested their benchmark on SOTA models.

[ Parent | on HN ]

Re:We built a real-world benchmark for AI code review by nomel on Wednesday February 04, 2026 @11:26PM (#46895669)

Whenever I see this, I make the (almost always correct) assumption that the SOTA models had an advantage, with the alternative explanation being a complete lack of awareness of the state of AI (which is very very rare for a tool like this).

With SOTA missing, it also is a strong indicator that someone like you is not the target audience.

[ Parent | on HN ]