I suppose it depends if there is some types of poorly placed barriers. In structures that can breath readily and in climates where they can dry it is less of an issue. On other hand those also need to in general be protected from water in structures.
A lot of that is due to structures not having a proper vapor barrier, therefor allowing vapor exchange from the outside world which decreases in-wall condensation.
Interesting whether this would happen in an entirely unheated building.