Re:The Era of 1-bit LLMs: ternary parameters for cost-effect...
by dns_snek on Wednesday February 28, 2024 @05:27PM
(#39544281)
It often fails at basic 3-4 digit arithmetic. If you're stretching that definition far enough to claim that GPT4 can "do math" then I should be able to call myself a commercial pilot because I can land a plane in a sim 20% of the time.
I'm not moving goalposts, the original claim was that LLMs can "do math". Primary school arithmetic is math.
GPT-4 can't do math and that's okay, I don't understand why so many of you are so touchy and defensive about this. It's a limitation that exists, nothing more, nothing less.
But whether or not it can "do maths" to your definition depends very much on what you want it to do, and how you define "do maths". To me it's irrelevant if it's doing the low-level calculations as long as it knows how to express them as code. If I wanted a calculator I'd use a calculator. And I don't consider a calculator able to "do math" just because it can precisely add numbers.
Meanwhile I've had lengthy discussions with GPT about subjects like orbital mechanics and calculating atmospheric effects where it correctly used maths that I had to double-check not because I didn't trust GPT (though I also want't to verify for that reason) but because I didn't know the maths (not that it was anything particularly advanced, but I lost interest in maths during my CS degree and picked the minimum amount of maths I could get away with).
By my definition it can "do maths" just fine. I guess you don't consider my view of that "reasonable". I can live with that, as meanwhile, it will keep doing maths for me when I need it.
Of course this was also a case of moving the goalposts to set up a strawman - in the comment of yours I replied to, you claimed it couldn't reliably add two numbers.