It's a matter of better governance though. Ideally, anti-trust government entities should monitor how companies like PayPal operate.

So, "ideally", we'd

1) ban cash, so that humans need permission to interact when an exchange of value is involved;

2) improve the oversight of the permission-givers, so that the permission-giver-watchers are less arbitrary and annoying, in order to fix the problem that never existed before point 1.

Other than lining the pockets of poor, underfed bankers and giving busy-bodies veto power over our interactions with others, what problem are we solving again?

I believe that the main reasonable concern with large transactions of money is mostly buyer/seller protection, for large sums of money. When buying, say, a house/car and spending well over €/£/$ 3k (typical over-3k transaction), it is customary to have a series of formalities/assurances so that both parties are protected against fraud. These formalities require the kind of proof (that money exchanged hands) that generally only banks can provide.

I fully agree that this places a big amount of power in the banks' hands, but what alternatives are there? (I'm honestly asking, are banks the lesser/only evil here?)

I'm not seeing cryptocurrency fixing this, but maybe it is possible. From my limited understanding, if transactions were mapped to some contract, it would void some anonymity of the system, and it probably wouldn't be possible to get money back in case of fraud anyway, so there's that.

Do we have anything that provide the best of both world?

I don't trust the governments and nanny states. Does that qualify as anti-trust? Better governance is lack of it, lack of corrupt government officials that work hand in hand with the financial institutions that sponsor them to take our freedoms away for their mutual benefit.