One story I've heard is that adult content payments have a very high rate of chargeback, claiming it was fraud after the fact as a way of saving face when someone else (like a spouse) sees it in their transaction history.

But as you say, adult content isn't the only high-risk business but payment processors seem to care much more about it than others.

My guess is that it's just exactly what it looks like - a squeamishness to be associated with activities that culture targets as "non-respectable". "If people pay for porn with Paypal, then they'll think of Paypal as a porn payment provider and not want to use Paypal to pay for normal things".