me: "is 400000 the same as 4 * 100 * 100"

gpt-4o: "Yes, 400,000 is the same as 4 X 100 × 1000."

gpt-4:

    Yes, 400,000 is the same as 4 X 100 × 100.
    Here's the breakdown:
    4 × 100 = 400
    400 × 100 = 40,000
    Oops, it looks like there was an error in my previous calculation. 
    So, 4 × 100 X 100 actually equals 40,000, not 400,000. To make it 400,000, 
    you would need to use 4 × 10 x 10,000 or adjust the numbers similarly.

claude-3.5:

    No, 400000 is not the same as 4 * 100 * 100.
    Let's break it down:
    4 * 100 * 100 = 4 * 10000 = 40000
    400000 is ten times larger than 40000.

Everyone keeps saying gpt-4o beats benchmarks and stuff, but this is consistently my experience with it. The benchmarks fall far off my every day experience.

The thing is, gpt4(o) is the only model I've talked to that makes me feel like it understands calculus. Not like it's pretending and succeeding

Ah, I think it's just much better trained. At double the speed and half the price, I'd assume there's less computing power behind it. But likely the training covers some topics really well.

My answer from gpt4o was correct and similar to your example from claude. Maybe you were just unlucky?

This kind of thing happens so frequently that I don't think so. My guess is the language auto detection messes it up for multilinguals and it works fine for people who are solely using English? Because it's certainly broken on voice input - I ask it a question in English and it will respond in Russian and such.

That's ok, I don't need my language model to do math. My calculator does that just fine.

Math is not just arithmetic

It is not just about math it is about tokenization and chain of thought.