Re:AI benchmarks are a bad joke – and LLM makers are the one...
by riskable on Saturday November 08, 2025 @01:39PM
(#45858863)
Nobody really knows "the point" of LLMs yet. They weren't even "invented" as much as they emerged as a trick to get computers to better understand human language.
They're still brand spanking new and everyone's trying to figure out how to best use them. We don't even really know if they're ever going to be "really good at" any given task!
Are they "really good at" these things or are they merely "OK-ish"?
* Answering factual questions.
* Programming.
* Understanding what the user wants from natural language.
* Searching/recommending stuff.
Real world testing suggests that with billions and billions of dollars spent, you really can get an LLM to be "OK-ish" at all those things :D
Re:AI benchmarks are a bad joke – and LLM makers are the one...
by BolexNOLA on Saturday November 08, 2025 @07:32PM
(#45861687)
> Nobody really knows "the point" of LLMs yet
Yet literally hundred of billions of dollars are being invested them…that’s what’s so concerning. And I can tell you not one of these startups would EVER acknowledge the truth of your statement.
The way they’re being deployed it feels like the point of LLMs is largely to replace basic online search or to run your online customer support cheaply.
I’m a bit out on a limb here because this is not really my technical expertise by any stretch of the imagination, but it seems to me these benchmark tests don’t really tell us much about how LLM’s perform in the ways most people actually use them. Maybe I’m off base here though