> Crater is a tool for compiling and running tests for every crate on crates.io (and a few on GitHub). It is mainly used for checking the extent of breakage when implementing potentially breaking changes and ensuring lack of breakage by running beta vs stable compiler versions.

https://rustc-dev-guide.rust-lang.org/tests/crater.html

https://github.com/rust-lang/crater

https://crater.rust-lang.org/

Even then such a bug as GHC's can conceivably not be detected.

It would be much harder to achieve this with GHC since the package manager is far from deterministic.

I'm not sure what you mean by this. `cabal-install` is as quite deterministic. The largest challenge in this area is the fact that core libraries' interface sometimes change. Our [head.hackage](https://gitlab.haskell.org/ghc/head.hackage/) infrastructure is one way of addressing this. I agree that it would be great to leverage `head.hackage` to test GHC against a larger body of packages.

It definitely _can_ be deterministic, but my experience of pulling down packages without a frozen index is that it rarely compiles. This, inevitably is because the plan depends on the current state of hackage. A combination of “it might pick a different minor version”, “different versions can cause cascading effects”, “many libraries have breaking changes” and “Haskell is extremely unforgiving of potential breaks” makes it very difficult to use without either a) being really good or b) downloading a freeze index. (I download freeze indexes.)

The whole core libraries thing is difficult, in that I can see it’s a real problem for everyone constantly updating their packages (and consuming them), but also that most of the changes are actually good ideas.

Yes, “crater” is the name of the tool.