Let’s say for the sake of argument that 500 of the candidates have the prerequisite programming ability to actually do the job, which they can figure out with a very easy coding exercise, because that’s what the actual job requires.

To filter out the other 499, they can either ask a very difficult question, which based on the previous paragraph we have established is not a useful signal, or we can use a random number generator. Asking a very hard question also rewards luck in a different way, because a candidate who is lucky enough to have recently memorized that question or a similar one will be able to pass it. It does not generalize to anything actually useful for the job.

I submit that the random number generator is a faster and cheaper method.