Some coins (PoS specifically) allow staking, which allows you to set it aside a certain amount, which is then used to validate other transactions, and you earn rewards. In traditional banking, this is kind of like a CD, and your money is used to help the bank out and it pays you interest on that.
The Ethernet (and a few other coins Solana comes to mind) ledger allow for the creation of smart contracts (applications that run on the blockchain) that could (probably some already exist) allow you to automate the creation loans on the ledger, witnessed by the world, that automatically pay you back. You can probably use the smart contract to do a modicum of due diligence on the borrower. But instead of paying SynapseFi (or other but first name that came to mine) thousands a month to allow you to build out loans, its all there for you on the block chain. This probably IS the future of peer2peer lending, as it is already a HUGE industry, and this would kind of get rid of the middle man.
One aspect of banking that is not really needed anymore would be the storage of assets. That is a built in part of cryptocurrency.
There is still the question of how do you get crypto, and for now, and until more people use it for everyday actives, that requires exchanges. And Exchanges could be seen as a centralization of sorts, but even they are a dime a dozen so they are effectively decentralized. And most support transferring to a wallet. So... kinda decentralized.