> To make this even worse, it may even produce Y just enough times to make it seem reliable and then it is unleashed without supervision, running thousands or millions of times, wrecking havoc producing Z in a large number of places.

Reminds me of the famous Xerox scanning bug: https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...