In Europe it seems we have them?
There are quite a few app-only or direct-banking things that opened up in the last decade (off the top of my head: N26, Monzo, Revolut, Starling, Illimity all have banking licenses).
It's true they often rely on some established bank but they haven't folded yet.
I've heard similar things from other sources. AIUI it's really hard to get a new banking license, both in terms of what you have to do to prove you are following regulations, and what you have to do to get the license.
In EU, I think one of the requirements is a minimal capital requirement. I can't remember the exact number, but something like 5 million EUR.
I don't think most of the small local banks could have started in the current environment, as I gather many of them were founded on a small community pooling their money to help each other and earn some money in return.
I don't know the details but the folks behind Simple tried, really tried to do the complete stack themselves. They gave up and first had a "real" bank handle it and eventually folded. There is a lot of red tape around creating a bank and as far as I understand, small local banks exist by being small and local. Like they couldn't operate nationally if they wanted to...