Re:Gemini 2.5 Flash
by omneity on Thursday April 17, 2025 @04:48PM
(#43722075)
In my case, I have workloads like this where it’s possible to verify the correctness of the result after inference, so any success rate is better than 0 as it’s possible to identify the “good ones”.
[ Parent
| on HN ]
Re:Gemini 2.5 Flash
by nonethewiser on Thursday April 17, 2025 @08:10PM
(#43723612)
Aren’t you basically just saying you are able to measure the error rate? I mean that’s good, but already a given in this scenario where hes reporting the 7% error rate.
No. If you're able to verify correctness of individual items of work, you can accept the 93% of verified items as-is and send the remaining 7% to some more expensive slow path.
That's very different from just knowing the aggregate error rate.
Re:Gemini 2.5 Flash
by yjftsjthsd-h on Friday April 18, 2025 @04:31PM
(#43731617)
No, it's anything that's harder to write than verify. A simple example is a logic puzzle; it's hard to come up with a solution, but once you have a possible answer it's really easy to check it. In fact, it can be easier to vet multiple answers and tell the machine to try again than solve it once manually.