Linking is often the slow part for large projects. Switching to the Mold linker can help some, but it can still be relatively slow on a lot of hardware.

Overall I agree with you though, it isn't slow enough on my hardware to interrupt my dev-flow.