This is a real problem and ventilating often helps, but depending on the air humidity and walls you have to heat above a certain temperature, yes.

I lived for years too poor to heat. Mold is a real issue. No amount of cleaning and ventilation makes up for not heating your home.

In my last year of study I was writing up my honours project and I had to stop to wipe the condensation off of my monitor. Mind you, it was about 2am.

I stayed overnight in an old castle. The owners didn't have enough money to keep it for a while in the water damage it caused was ridiculous. Castles are dependent on fires in every room to drive the leaking water out.

You can probably not avoid heating your home completely (unless you constantly have a window open), but you can lower the temperature by a couple of degrees without having mold problems if you also keep an eye on humidity.

Dehumidifying & ventilating will be more effective anyway.

dehumidifier. no need to heat

Total game changer. Pouring out a tank full of water that otherwise would have gone into your walls is still a delight. And the waste heat warms the room up too.

That is what I have in my shed. It doesn't work because the water freezes to the coils. Once this starts it keeps getting worse until you thaw the coils. Sure if you live where the temperature is never below freezing they are an option, but when the temperature gets colder they are not an option.

Not to mention I need heat to keep my plumbing from freezing.

Check out rotary desiccant dehumidifiers. They can work at very cold temperatures. They work by continuously regenerating a desiccant disk.

If you're UK based, I can recommend the EcoAir line of dehumidifiers. (Not affiliated).