One theory I heard about this type of problem is because these algorithms tokenize the text early, and each token can be multiple characters.
Re:Claude 3.5 Sonnet
by logicchains on Thursday June 20, 2024 @02:25PM
(#40741647)
This isn't a very good question because of how tokenizers work: it can't see individual letters. If you wrote it with some separator between the letters, e.g. r - a - s - p - b - e - r - r - y, it'd probably do much better.
The only one that got it right was the basic version of Gemini "There are actually three "r"s in the word "strawberry". It's a bit tricky because the double "r" sounds like one sound, but there are still two separate letters 'r' next to each other."
The paid Gemini advanced had "There are two Rs in the word "strawberry"."