Their overall reaction seems way out of proportion here, I agree. But the general tactic of questioning someone seems sorta like the entire point of having these agents in the first place.
Some airports in EU countries (eg Shannon in Ireland) have pre-clearance for passengers to the US at the departure end, ie you go through US passport control first and when you get on the plane it is legally a domestic US flight. This is unusual however, just as Americans typically deal with EU passport control at the EU end rather than before leaving the US.
If you want to avoid dealing with these problems in the airport, the solution is to apply for a visa from the US embassy before departure.
When you try to go to the US, you have to get in some kind of trap (there are those stupid questions which I guess are a framework to make you a liar to a federal agent if you get caught later), drop all your privacy (that was probably gone anyways with facebook and gmail), and they have ample means to close it on you without being an asshole.
Sorry, but I think you're projecting your biases here with comments like 'they chose to let him come, knowing they would refuse him.' My reading is that he probably answered 'no' to all the ESTA questions and then confused the CBP agent by giving contradictory answers to his questions. I'm European myself so I'm not carrying a particular torch for US border agents here.
Do you mean, the US should open border offices in all international airports? :)
But even if he did made a mistake wouldn't it be easier to inform him before he enters the plane? And avoid interrogating him without letting him know what the problem was? The process he got ranks pretty high on the crazy paranoid side. I'm sure there are ways to solve the security and visa issues in a way that does not leave traumatic memories in every casual traveller that maybe or maybe not misunderstood the rules.
It's obvious that the border control knew what the problem was. Why not tell him the problem in a polite way? Or why is there no process that avoids that the person has to fly overseas in order to be scared and not allowed entry? I'm sure there are legitimate security related cases where such a behaviour would not catch a few illegal immigrants but all these stories read like they tuned their ROC curve to maximize the true positives and forget about false positives at all. It's the same idea behind the NSA spying an the no fly lists. A huge false positive rate no matter what the costs are.
Looks like this decisions are not the result reason and careful consideration but instead full paranoia mode. But it's easy to say that from the outside. We don't know.