That's really the big reason why savings rates vary so much between banks, if you have a bunch of consumer credit lines then high savings rates make sense, if you have a bunch of lower fixed-rate collateralized loans then you can't get away with lowering your spread so much.
There's other epicycles to this too. For example, a bank can just "buy" deposits through a correspondent bank (a bank-for-banks). So the direct connection between your kind of borrowing and kind of lending, particularly when its consumer / credit card stuff, can be tenuous. In SVB's case, though, all of the stuff happened in a tight echo chamber ecosystem -- SVB got deposits from tech cos, who got money from VCs, who were working off of capital call LOCs from SVB. So your observation is germane.