Re:AI benchmarks are a bad joke – and LLM makers are the one...
by cyanydeez on Saturday November 08, 2025 @06:06PM
(#45860867)
Id say your experience is being more monetized for growth for growth sake.
[ Parent
| on HN ]
Re:AI benchmarks are a bad joke – and LLM makers are the one...
by bjackman on Sunday November 09, 2025 @06:32AM
(#45864811)
Actually I disagree that that's what's going on in the world of hyperscaler platforms. There is genuinely a staggering amount of money on the line with the efficiency of this platform. Plus, we have extremely sophisticated and performance-sensitive customers who are directly and continuously comparing us with our competitors.
This isn't just that nobody cares about the truth. People 100% care! If you actually degrade a performance metric as measured post-hoc in full prod, someone will 100% notice, and if you want to keep your feature un-rolled-back, you are probably gonna have to have a meeting with someone that has thousands of reports, and persuade them it's worth it to the business.
But you're always gonna have more luck if you can have that meeting _before_ you degrade it. But... it's usually pretty hard to figure out what the exact degradation is gonna be, because of the things in my previous comment...