I don't know the details but the folks behind Simple tried, really tried to do the complete stack themselves. They gave up and first had a "real" bank handle it and eventually folded. There is a lot of red tape around creating a bank and as far as I understand, small local banks exist by being small and local. Like they couldn't operate nationally if they wanted to...
There are quite a few app-only or direct-banking things that opened up in the last decade (off the top of my head: N26, Monzo, Revolut, Starling, Illimity all have banking licenses).
It's true they often rely on some established bank but they haven't folded yet.
I don't think most of the small local banks could have started in the current environment, as I gather many of them were founded on a small community pooling their money to help each other and earn some money in return.
This would also raise the question: If the current low costs of retail deposits allow banks to print money, then why aren't we seeing more banks open up to print money?