Reasoning Is Not Model Improvement

Re:Reasoning Is Not Model Improvement by rahimnathwani on Friday October 24, 2025 @05:10PM (#45699138)

  Let's compute exactly. 87,654,321 * 98,765,432.

  So answer: 8,657,216,880,631,672.

Seems like it got one digit wrong. Python tells me the answer is:

8,657,216,880,231,672

I asked gpt-oss-20B the question three times. It took different routes each time. The first time it made some mistakes early on and then spent ages getting more confused. The other two attempts were successful.

[ Parent | on HN ]

Re:Reasoning Is Not Model Improvement by alganet on Friday October 24, 2025 @08:32PM (#45700455)

Damn, I'm kind of ashamed that I haven't checked the final result myself. I assumed simonw checked it before posting and blindly went with it.

[ Parent | on HN ]

Re:Reasoning Is Not Model Improvement by simonw on Friday October 24, 2025 @09:42PM (#45700744)

Wow, good catch! I eyeballed the first few digits and the last few digits and they were a match so I assumed the digits in the middle were correct too.

[ Parent | on HN ]

Re:Reasoning Is Not Model Improvement by rahimnathwani on Friday October 24, 2025 @10:03PM (#45700836)

I have a 9yo son and have learned to verify each digit :)

[ Parent | on HN ]