You're absolutely right that the savings on interest is post tax. However the tax deductibility of interest creates a substantial tax shield and in general, has a net positive value for the company since changing the capital structure by increasing debt decreases the weighted average cost of capital. Studies suggest that debt, because of the deductibility of interest, on average leads to ~1 dollar of additional company value for every 10 dollars of debt financing (vs equity financing).