It really depends. The air that you exhale is hot and humid. As soon as it touches the exterior walls, it will condensate. I'm talking about a house that is not insulated, but also does not leak air out (new windows with good seals). If it leaks some air out, then the condensation is very limited as outside cold air is dry in winter.

It might be like you said only if you don't use that room, or you have a big house so maybe you don't use most rooms and then you don't fill them with humidity from your own breath. Walls are cold but also dry.

Opening the windows in the winter is how you let all that humidity out.