Another point is that the idiom "a penny saved is a penny earned" is incorrect when it comes to loans and investments, because you pay tax on income. A penny saved is worth more than a penny earned.

If I borrow at 4% and invest the same sum at 4%, then I'm guaranteed to lose money on the deal. How much more your investments have to yield, in order to break even on a loan-to-invest strategy depends on the degree of tax liability that your investments are subject to.