News for hackers. Stuff that matters.
Re:Benchmarking Coding Agents on Databricks' Multi-Million L... by falaki on Wednesday July 08, 2026 @08:27PM (#48839352)
Also they suggest every company should build their own benchmark and repeat these tests with new models instead of relying on the SWE bench.
Re:Benchmarking Coding Agents on Databricks' Multi-Million L... by fmind-dev on Thursday July 09, 2026 @03:17AM (#48842080)
It takes time and effort to build such benchmark. It works at Databricks scale, I'm not sure smaller companies are ready to invest on internal benchmarks.

But they are more vendor neutral, now they don't sell their own model. It's interesting from a benchmark point of view.