News for hackers. Stuff that matters.
Re:AI benchmarks are a bad joke – and LLM makers are the one... by anamexis on Saturday November 08, 2025 @12:37PM (#45858429)
Computer vision algorithms run on computers and they can’t do basic arithmetic.

My email client runs on my computer and it doesn’t do basic arithmetic either.

Something running on a computer does not imply that it can or should do basic arithmetic

Re:AI benchmarks are a bad joke – and LLM makers are the one... by TheOtherHobbes on Saturday November 08, 2025 @01:01PM (#45858584)
That's confusing basic arithmetic as a user feature and as an implementation requirement.

I guarantee that computer vision and email clients both use basic arithmetic in implementation. And it would be trivially easy to bolt a calculator into an email app, because the languages used to write email apps include math features.

That's not true of LLMs. There's math at the bottom of the stack. But LLMs run as a separate closed and opaque application of a unique and self-contained type, which isn't easily extensible.

They don't include hooks into math features on the GPUs, and there's no easy way to add hooks.

If you want math, you need a separate tool call to conventional code.

IMO testing LLMs as if they "should" be able to do arithmetic is bizarre. They can't. They're not designed to. And even if they did, they'd be ridiculously inefficient at it.

Re:AI benchmarks are a bad joke – and LLM makers are the one... by anamexis on Saturday November 08, 2025 @01:04PM (#45858615)
Yes, you are agreeing with me.
Re:AI benchmarks are a bad joke – and LLM makers are the one... by gishh on Saturday November 08, 2025 @01:05PM (#45858620)
Pretty sure the only thing computer vision does is math.

I’ve also observed email clients tallying the number of unread emails I have. It’s quite obnoxious actually, but I qualify adding as math.

Re:AI benchmarks are a bad joke – and LLM makers are the one... by ghurtado on Saturday November 08, 2025 @02:32PM (#45859270)
> Pretty sure the only thing computer vision does is math.

That is only marginally less pedantic than saying that the only thing computer vision does is run discrete electrical signals through billions of transistors.

Re:AI benchmarks are a bad joke – and LLM makers are the one... by gishh on Sunday November 09, 2025 @03:16PM (#45868777)
If you’ve ever written code for a computer vision application, you’d realize how incorrect this statement is.
Re:AI benchmarks are a bad joke – and LLM makers are the one... by anamexis on Saturday November 08, 2025 @01:11PM (#45858666)
Yes, everything that a computer does, it does using math. This does not imply that things running on the computer can do basic arithmetic tasks for the user.