News for hackers. Stuff that matters.
Re:Cargo Cult AI by jacquesm on Saturday August 05, 2023 @01:26PM (#37014335)
That could still be explained by that example having been part of the training set so it knows how to give the right answer. But there must be more going on than that:

The square root of 232444232 is approximately 15229.

(which is wrong, it is approximately 15246.12)

The sum of 2341347345 and 234823542354 is 237165889699.

Which is the right answer.

So there may be some special casing happening there.

Re:Cargo Cult AI by bcherny on Saturday August 05, 2023 @01:28PM (#37014359)
I mean, I don’t know the square root of 232444232 off the top of my head either..
Re:Cargo Cult AI by blibble on Saturday August 05, 2023 @01:31PM (#37014403)
if you've had a high school education presumably you could work it out

it can't

Re:Cargo Cult AI by jacquesm on Saturday August 05, 2023 @01:33PM (#37014429)
I can. Newtons method is pretty easy to do in your head, but with larger numbers you need to be very careful not to mess it up. But on paper it's trivial.
Re:Cargo Cult AI by sillysaurusx on Saturday August 05, 2023 @01:40PM (#37014523)
In other words, you need an external tool (paper) which GPT doesn’t have.
Re:Cargo Cult AI by kortilla on Saturday August 05, 2023 @01:49PM (#37014623)
Yes, GPT has access to memory.
Re:Cargo Cult AI by jacquesm on Saturday August 05, 2023 @02:33PM (#37015041)
No, I actually can, just not for arbitrarily large numbers. And 'paper' is just temp storage, not a tool. A tool in this case would be a sliderule or a calculator or an abacus. Long division requires pen and paper too if you want to 'show your work', just like I would have to show on paper how I did the square root of say 47515, you could simply choose not to believe me if I spat out the answer, but if I showed you step-by-step on paper and you followed every step you would either also conclude that it is correct, or alternatively that I had made a mistake.

That's why I think it is significant that chatgpt gets the addition spot on but gives a wrong answer to the square root problem. I can do better than that off the top of my head and I do not have access to the same computational resources that it has.

Re:Cargo Cult AI by eis on Saturday August 05, 2023 @01:38PM (#37014501)
One difference is that you are aware that you can't do it and state so. Our current LLMs will just give whatever result they think it should be. It might be correct, it might be off by a bit or it might be completely wrong and there's no way for the user to tell apart from double checking with some non-LLM source wich kinda defeats the purpose of asking the LLM in the first place.
Re:Cargo Cult AI by sarchertech on Saturday August 05, 2023 @01:44PM (#37014577)
So you'd either work it out and check that it was right. Or you'd tell the person asking that you didn't know. You wouldn't just make a plausible answer and confidently state it. If you did that frequently, people would stop listening to you.
Re:Cargo Cult AI by pixl97 on Saturday August 05, 2023 @04:03PM (#37015810)
What does the word approximately mean, if we go back to the previous statement answer from GTP 4, and what precision is needed in the answer?

And, no I'd grab a damned calculator and let the specialized tool do the work, which it turns out if you turn on plugin mode GPT-4 can use the same tools and get an exact answer.

Re:Cargo Cult AI by sarchertech on Saturday August 05, 2023 @07:45PM (#37017390)
That number isn’t correct to any precision and the fact that there wasn’t rounded to the nearest 10, 100 etc… means that no human would recognize it as an approximation to anything but the nearest whole number (which it wasn’t).

>calculator

Well in this case you don’t have access to a calculator because there was no plugin. And you likely wouldn’t just make up a number like chatgpt did.

This perfectly illustrates the danger of chatgpt. It will confidently make up an answer to any question whether or not it has access to specialized tools in a way that most humans won’t.

Re:Cargo Cult AI by kenjackson on Saturday August 05, 2023 @02:05PM (#37014786)
Actually 15229 is a decent approximation. It’s a better approximation than the one I did off the top of my head.
Re:Cargo Cult AI by jacquesm on Saturday August 05, 2023 @02:37PM (#37015071)
You need more practice :)

Simple trick: divide by 100, 10000 or 1000000 use Newtons method on the integers, then multiply by 10,100 or 1000 and add a 'fudge factor' based on how large the fraction was...

It's cheating but it can get you pretty close, I'd peg it at 15240 using that trick. If you just want to do the closest squares you can average between 15 (too low) and 16 (too high) so you'd guess 15500, which is much too high, but one more iteration of Newtons method gets you closer than what chat gpt gives.

Re:Cargo Cult AI by sarchertech on Saturday August 05, 2023 @07:55PM (#37017469)
And if chatGPT said “I don’t know the actual answer but my best guys is 15229” that would be a reasonable and potentially useful answer.

The fact that it gives you a number that isn’t rounded to the nearest tens, hundreds, or thousands place means that it doesn’t look like an approximation to any reasonable person, which makes it a terrible answer.

My younger brother used to have this problem. If you ask him a question like “how long until you get here”, he’d say “17 minutes”. What he really meant was “around 20 minutes”, but everyone thought he must know the exact time. Like he’d done the drive many times, or he was looking at his GPS.

Re:Cargo Cult AI by andrepd on Saturday August 05, 2023 @02:27PM (#37014985)
> So there may be some special casing happening there.

100%. Maths was a notable weakness of earlier GPT versions, so ChatGPT-4 has a layer to direct mathematical queries to an evaluator.