GPT-4 can do some reasoning, just not at a smart/trained human level. You cannot explain its results in USMLE, Uniform Bar Exam, and several other tests otherwise. This is a major improvement from earlier models.

This assumes there is not too much data contamination, but other people’s experiences with it suggest that the capability is real even if it might be a bit below what the test performance suggests. The latter is also explainable by a broader distribution of real world problems relative to tests.

See my other recent comments for more examples and elaboration, eg https://news.ycombinator.com/item?id=35301499