Critical thinking is domain dependent. For example, just because you can kill it on white board interviews doesn't mean you will be able to navigate relationship issues. The way I imagine many PhD's I've talked to is like the RPG character where you max out one line of development at the expense of all the other lines. The counter-example would be the "T shaped" development profile which is the ideal imo.

A research PhD should be able to instantly recognize logical fallacies outside their field, but in my experience they fall for them every time.

Absolutely!

I often wonder whether the epistemology we are "born with" can actually be improved very much (in a general sense), or whether the best we can do is teach domain-specific techniques to override our defaults on those topics.

We all fall for it. Just search for: biases

We also need to deeply care and understand ourselves.

INT and WIS are separate, orthogonal stats