Taxes certainly complicate that, but most people do what you're describing. People with moderate and even high net worth (that is, in financial assets) still use mortgages because it's better to borrow at 5% instead of liquidating an asset that is returning 9.7%.