I figured out the issue: I transplanted the bit about setting optimization levels from the blogpost into my setup (which sets opt-level = 3 for all dependencies), and when I removed that, my compile times went down by a factor of 5-10!