That said most people plan to do this, and never get round to it.
I knew I was making a "mistake" when I started putting all my spare money into paying off my mortgage early, in the sense that I could have achieved more money by investing. But it seemed obvious to me that owning my house would let me ride out problems more easily in the future - my bills would suddenly come down to "electricity, food, and little else".
As it happens my wife and I decided at reasonably short notice to change countries. Having paid off my mortgage allowed me to rent out the place, without having to worry about getting permission from the bank, or anything similar. I just had to register as a landlord and switch to a more expensive form of insurance.
After two years of being abroad selling the place made sense, to avoid complex cross-country taxing issues. But by owning the place outright we had the freedom to try renting for a while, and worry less about income.
Rid yourself of "bad" or "marginally bad" debt immediately, manage the "good" debt, then save (rainy day fund, 401K, IRA, speculation). You're doing well if you get to the speculation part.
The conspiratorial part of mind suggests that finance is more invested in having people in debt than without, and the math is there to beat people over the head saying "you are not playing this game well because this math says you will lose in the end"