Thanks for this comment. This highlight the crux of the issue. Banks should have something like "banking neutrality" where they'll bank anyone and any money until a court order says otherwise.

It's funny most people accept this situation which can significantly abused by lots of parties (See: Operation Choke Point as an example). But if you live in a small village, and there is only one bank branch, the banker there not liking you means you being unbanked for no reason.