Because they are? Approximately zero risk, true, but slightly over zero. Its a pretty simple straightforward game theory problem. Whats the total aggregate sum of the trouble you'll get into by letting 1 out of 100 million terrorist thru, vs simply stamping "denied national security" on any random visa and stonewalling? In fact the guy who dared complain about it is probably now on some secret NSA list, and perhaps also now on the no-fly list. That'll show people what happens to complainers, hopefully that prof never has to fly anywhere again for the rest of his life... and probably his families lives, too.
Note, I'm not supporting an unjust system, just explaining how it works from a game theory perspective. I don't see why any individual inspector would ever permit a visa unless there's some kind of piecework payment system or a really good non-retribution system (anonymity?). If a terrorist ever got past them, there would be demands for punishment. Therefore "the system" is going to select for inspectors who are either reckless, careless, actively working against the interests of the USA... None of which will probably have ideal outcomes.
The current process acknowledges the fact that there will be a number of false positives, probably an order of magnitude higher than true positives - a class of people which the system aims to filter impeccably.
However, we should not forget about the sentiment this process brings about. It leaves a very poor taste and even hurts and insults a lot of people and the cost of losing those talent and taxes thereof could be manifold.
You'll understand only when you face it. What legitimate reasons are you applying to justify treatment of any or every person on the planet as potential threat? Without a means for recourse. And then if you're agreeable to the logic of scrutiny and transparency from others then why not apply the same logic to the individuals in the Government?