The main thing we're trying to show is that Mo finds over 3x as many correct bugs in the same amount of time as the next-best QA agent. This is the main thing we optimize for since the worst thing you can do in QA is miss a real bug and ship broken code to prod. Currently, the tradeoff you make for that better recall is higher cost. I would say the precision is more or less comparable, but marginally lower.

Obviously we'd love to be beating every other agent on all of these numbers, and we're actively working on it, but for now this is an honest and transparent look at how the tool is doing.

Feel free to take a look at the full writeup for more detail!

I also do believe search quality is recall dominated, but the link between the metrics and results are not as clear as they could have been. I think you have a stronger sell here than it looks at first glance.

Congratulations to the team tho! Those are very nice results

Maybe it varies by domain but I'm quite used to codebases where "fix every discovered bug" is basically infeasible; finding 1000 miniscule bugs isn't helpful. Finding _serious_ bugs is the goal.