Agents need control flow, not more prompts

Re:Agents need control flow, not more prompts by jiehong on Friday May 08, 2026 @02:27AM (#48059375)

This might be inherent to how the models are benchmarked.

Aren’t some benchmarks giving the model multiple shots at a problem and only keep the successful result if it appeared, ignoring the failure rate?

[ Parent | on HN ]

Re:Agents need control flow, not more prompts by andyferris on Friday May 08, 2026 @02:34AM (#48059422)

Good point. We need the mean, “any 1 of 10” and the “all 10 of 10” success rates in the metrics, so we can estimate reliability (the last one).

[ Parent | on HN ]