News for hackers. Stuff that matters.
Re:GPT-4.5 or GPT-5 being tested on LMSYS? by imtringued on Tuesday April 30, 2024 @03:35AM (#40208202)
Me: 478700000000+99000000+580000+7000?

GPT4: 478799650000

Me: Well?

GPT4: Apologies for the confusion. The sum of 478700000000, 99000000, 580000 and 7000 is 478799058000.

I will be patient.

The answer is 478799587000 by the way. You just put the digits side by side.

Re:GPT-4.5 or GPT-5 being tested on LMSYS? by Sunhold on Tuesday April 30, 2024 @09:23AM (#40210718)
Re:GPT-4.5 or GPT-5 being tested on LMSYS? by jiggawatts on Tuesday April 30, 2024 @06:57PM (#40217484)
I recently tried a Fermi estimation problem on a bunch of LLMs and they all failed spectacularly. It was crossing too many orders of magnitude, all the zeroes muddled them up.

E.g.: the right way to work with numbers like a “trillion trillion” is to concentrate on the powers of ten, not to write the number out in full.

Re:GPT-4.5 or GPT-5 being tested on LMSYS? by hxypqr on Wednesday May 01, 2024 @06:37AM (#40221553)
Predicting the next character alone cannot achieve this kind of compression, because the probability distribution obtained from the training results is related to the corpus, and multi-scale compression and alignment cannot be fully learned by the backpropagation of this model