And if your excuse is "well, we're too big to know our customers!" maybe you shouldn't be in the business of loaning money.
If you're an immigrant? If you're a protected class?
The whole point of the fair lending act is that lenders have to use consistent lending criteria across population groups. Going back to the pastoral days of personal relationships with bankers is the wrong answer.
PS "lending" not "loaning"
If you are using a different bank in this new town, then yes, you're kind of SOL. However, it's not too wild to think of a way for credit history and verified identity from one bank branch to be stored in some kind of software system, which could share data between branches. The initial work in establishing trust is annoying, but verifying identity once you have some system in place is easy enough. A short phone or video call with someone who _did_ know you and works within the same institution would be a pretty sufficient way to go about it. Hell, universities have an even more archaic way of doing this: if you need proof of enrollment or a transcript then the university will send sealed snail mail to whatever institution needs the information. You can even request and deliver these yourself personally, if you want to.
Perhaps "personal" is the wrong word for the kind of relationship you want to describe though. Your relationship with the bank as an institution is independent of any SSN, they just track any credit or debt to your identity using it. There's no reason why a single number needs to track that kind of information, when the banks could just be required to verify their own knowledge of you themselves. At some point we have to admit that someone at the bank must know you, or be able to vouch that who you are is who you say you are. Even just a second check of a photo of you compared to a face scan would be more than what's verified now.
Alice in Pittsburgh wants a $100K loan to go to college.
Bob in Hawaii wants to invest $50K at a 5% return rate.
Charlie in China wants to invest $50K.
Without banks - Alice & Bob fly across the country to meet, and Bob, who knows nothing about college, hopes the college is real & Alice will pay. Alice, who knows nothing about Bobs hopes he won't get really impatient after 10 years and rob her. They talk for 30 minutes and hope that short interview is enough to establish trustworthiness on both sides.
We then repeat the whole process with Alice & Charlie.
With banks - Bob invests $50K into the bank. Charlie invests another $50K. Alice gets a $100K loan from the bank. The bank already has channels to ensure Alice is trustworthy, to verify her admission/enrollment in college, to collect loan payments, to eat the loss and give Bob his 5% returns anyways if not repaid. Even though Alice pays back $12K every year, the bank has enough liquidity to give Bob $1K every month instead as he prefers.
By the way, banks don't lend deposited money, they create money.
Bob and Charlie get maybe 1% interest on their $100k, while the bank charges 10% on all $1000k of the leveraged loans. At year's end, the bank has paid B and C about $1k, and collected from A D E F G H I J K and L about $100k. If Bob, Charlie, or the bursar ever unexpectedly come around to make a withdrawal, the bank phones up the national central bank and says, "Yo. Boss says you need to come by and do that thing we talked about that one time. It's important." Then a truck shows up with very important pieces of fancy paper that the bank can give to people, so that they go away without getting mad. It's very important that no bank ever runs out of those, because then people wouldn't give them back to the banks to be reused, by giving them to other depositors!
It's all very complicated and important and not crooked at all. You can certainly trust those banks, because all their employees are clean and smell nice and speak clearly and wear good-looking suits.
But it is also important to realize that the bank does not need to know Bob or Charlie very well. It isn't really even strictly necessary that Bob or Charlie be real people. All that is necessary is that they have entrusted a symbol of their savings to the bank, to be used as the bank sees fit, without any direct oversight, in exchange for a pittance in interest. This is really only a good deal if you have so much money that your investments would otherwise overwhelm the capability of businesses to generate a good return on it. So banks have a few customers that deposit enormous quantities of cash with them, because the only other entities that can actually pay out a 3% return on that much dosh are national governments, state-partnered enterprises, and multinational mega-corporations. Bob and Charlie represent round-off errors. But there are a lot of Bobs and Charlies, and in aggregate they do help the bank make money, so they are tolerated, with a token amount of ass-kissing to keep them happy. And if one gets screwed over by accident once in a while, that's no big deal (to the bank).
The bank does need to know Alice and D E F G H I J K and L, but only just enough to figure out how much extra they need to charge to cover their default risk. The bank's major concern is that one of those people might actually be Maurice, who plans to default, but he is untouchable, because the bank does not know who he really is. But they know who he was pretending to be, so screw that guy; he shouldn't have let someone else pretend to be him without his knowledge, right?
So I'm not really certain that knowing all the insignificant peons would actually help. The fundamental problem with banking is that the (remaining) individual firms are too large for the median customer to matter to them at all. They have no intrinsic economic incentive to even acknowledge that a problem exists, much less admit responsibility for it, because their real business is aggregating investment opportunities that are too small to bother with individually into packages large enough to interest investors with colossal amounts of cash, and taking a percentage off the top.
What alternative would you propose now?