On the institutional side its the well known and agreed upon siege mentality where the church tries to protect itself out of arrogance and a probably accurate assessment that its enemies want to use and manipulate this for political gain.
Also much less talked about is the church's former policies were partially due to a misguided sense of mercy. If the church really did come down as hard and preemptively as people now want it to from the very beginning the world would probably be whining about how intolerant and cruel they were to potential and defrocked priests.
Priests have sexual needs, that's understandable, but they could satisfy them without victimizing others. The line is harming someone, specially if they are defenseless. The beliefs of the perpetrators don't enter the equation.
It seems entirely predictable to me that this drives selecting folks for these positions that don't want wives, but want something else.