Get rid of Banks and many different things change.

Some wealthy investors would setup a lending operation using their own cash, based on historical examples of such. They would however need to charge higher interest rates than banks because they would be more capital constrained.