For perf sensitive projects, I set opt-level=1 in debug for at least the dependencies.

Edit: I realized you might have meant that debug compiles still take a long time. In that case, obviously this won't help.