One piece of it is actually regulation. When you're dealing with huge amounts of yellow tape and you have millions of customers, the overhead cost of providing services is absolutely sky high. Many of their clients don't even make them money; think of the Starbucks barista that has $1000 in checking and no savings. Sure they can turn some of that money around into interest, but the cost-per-customer of lawyers, compliance, auditing, etc is far higher.

Any of the banks that are actually offering decent interest rates on deposits are online banks, because it enables them to centralize and automate all of their operations in one place.

Credit Unions are similar. They usually serve a smaller community, which means they're centralized and have less overhead. They are also beholden to different less-strict laws.

I've heard this first-hand from the founder/CEO of a small local bank. Every time they write a loan they are forced to provide the customer with a 200-page "book" of all the compliance and regulation surrounding that loan. The client immediately throws it in the trash, every time. It hurts the bank, because they have to print that, audit it, have lawyers look it over, maintain it. It also hurts the client; these laws are meant to inform consumers, but in reality they irresponsibly only require banks to throw incomprehensible amounts of information at consumers. Like website Terms of Service, consumers never read it, they just trust what the bank teller tells them.