How well does your LLM do?
You go to a shop and buy three things. You work out the cost as being £5.88.
When you get to the till you realise that instead of adding the items together on your calculator you actually pressed multiply each time instead. Interestingly the cost at the register also comes out to £5.88. What was the cost of the three items.
Try asking your preferred LLM to solve this? How did it do?
(
9 comments
| 2 points
| on HN
)
then works through it with the kind of faulty thought process that I'd expect from a student who squeaks through an intro physics class and gets a D grade.
It give me 2.7, 1.1, and 0.22 which don't multiply or add to 5.88.
I told it was wrong, it gave me some numbers that summed right but multiplied wrong, pointed that out and it gave
1.20, 1.50, 3.18 which sums right and multiples to 5.724 which is a little low.
If you are treating it as actual cents though, I don't think there's an exact answer as a/100,b/100 and c/100 will come out of the factors of 588 and I just don't see getting enough big numbers to add up to 588.
Phi-4 (unsloth's dynamic 4-bit version) keeps trying random and repeating numbers with no particular strategy. Adding Maharshi's "contemplative prompt" did not really change anything.