Of course, past Econ 101, we also get into issues of people making trades they think will be beneficial but aren't.
It comes at a cost, but a person and the bank might both come out ahead in utility if the person has a stronger preference for money in the short term than the long term (see "Time Value of Money", "Hyperbolic Discounting") compared to the bank. There are a number of forms this can take, but one worth calling out is when the person is able to use the money to produce the thing they are going to trade for more value.
Does all of this assume growth? Not really. It assumes someone is creating something somewhere, but it doesn't assume society is creating more tomorrow than today.
That said, even that is not assumed by Economics - because nothing in Economics says banks will always be willing to make loans. If conditions are such that they do not expect to be able to be paid back (often enough to have a positive expected value), Economics says banks stop lending. There is no assumption of infinity.