(AFAIUI Cabal:Custom had exactly the same reasoning, btw. "It's too difficult! Let's just give up and allow anything.")
It might be a practical reality that this is just unavoidable if you want to achieve success, but at a meta-level I think deserves thinking about whether builds should actually be allowed the full Turing Complete treatment. At least, theoretically: As long as we continue 'allowing' Custom for practical reasons and without serious thought, there can be no progress.
(Btw, I'm quite mad which it comes to this type of thing. I spend way too much time thinking about build systems, etc.)
EDIT: This problem manifests EVERYWHERE: Docker builds randomly guess wrong about whether inputs have changed, cabal-install gets it wrong, SBT/Zinc doesn't quite track dependencies correctly, etc. etc. and it's infuriating.