Re:LLMs can't self-correct in reasoning tasks, DeepMind stud...
by vidarh on Tuesday October 10, 2023 @01:59AM
(#37828956)
It knows the letters each word is made up when asked within a specific context. Humans also often struggle to recall things in one context that we have no problems with in another. It may well be reasonable to call it "too stupid" but at the same time it's unreasonable to call it stupid without then acknowledging that it can understand and reason. That it has gap in knowledge in areas we typically drill into young children and don't leave huge datasets online about is unsurprising to me.
That said, I incidentally think a whole lot of adults - even native English speakers - would struggle with the task given, and would repeatedly fail in the same way until given a detailed refresher.
E.g. being able to explain a rule and fail consistently to apply it is something I've seen up to and including supposed senior software engineers im interviews. Getting their mistakes explained and still repeating the same mistakes also.
That said, I incidentally think a whole lot of adults - even native English speakers - would struggle with the task given, and would repeatedly fail in the same way until given a detailed refresher.
E.g. being able to explain a rule and fail consistently to apply it is something I've seen up to and including supposed senior software engineers im interviews. Getting their mistakes explained and still repeating the same mistakes also.