I'd expect that OpenAI's stronger reasoning models also don't generalize too far outside of the areas they are trained for. At the end of the day these are still just LLMs, trying to predict continuations, and how well they do is going to depend on how well the problem at hand matches their training data.

Perhaps the type of RL used to train them also has an effect on generalization, but choice of training data has to play a large part.