I think there is more to it than pure compile time. It can affect responsiveness enough that it becomes disturbing.

From there it can mean using fewer plugins, disabling part of the static code checks, doing less full debugging etc.

I remember how activating the debugging flags wouldn’t work on some projects as the machine couldn’t handle it memory wise.

All of these have a compounding effect that is non trivial.