This is a simplification, but: the bank should have to offer depositors an interest rate (or other value-add, such as competitive services) that makes it worth their while to trust the bank with their deposits. A bank that makes very conservative investments/loans can only offer a low interest rate to depositors, because they earn less profit on the spread, but their depositors will accept it because of the safety of the institution. A bank that makes speculative investments/loans can offer a high interest rate to depositors, because they earn enough to offer it, and they'll need to, to attract them away from the conservative institutions.

A bank that can offer government-backed guarantee to depositors can offer a low interest rate while making big profits, and pay big dividends to their shareholders and executives, steal market share from their conservative competitors while making themselves more systemically essential in the process, and leave the public on the hook if they fail.