There are few compilers that are slower than GHC. I suspect this issue has nothing to do with Nix, and the tools would have taken ages to build from scratch without Nix.