We lower the heat if nobody is home. But, that's the difference between 70* and 65*. With no heat, the house would eventually be 32* in the dead or winter.

And we only do that because smart thermostats exist and use our phones as presence sensors. Without that, we'd have to manually lower the heat and I doubt we'd bother/remember.