There's also some question about the fragility of some of these models, compared to humans [1]. I think there have been some other examples where changing one rule makes their aptitude crash. The implication being they are still hard-rule based, but scale well. But that scale advantage erodes quickly with a rule change because of the lack of contextual understanding.

[1] https://arstechnica.com/information-technology/2022/11/new-g...