The problem is on the demand side. If consumers spend less (due to higher cost of borrowing), this makes it difficult for smaller businesses to survive or raise prices (while bigger businesses have access to cheap loans/can reduce workforce etc.).