That is my current family plan, no mortgage (since rent is cheap and it does not tie you down a lot of it is mental rather then purely monetary). For kids, I am not slaving away so they can party at college for 4 years. Disclaimer. I do not have kids yet(have wife) so who knows maybe they screw with my head enough to change my mind. I have enough example of nutty parent friends who whine about how their work sucks but they got to have the paycheck!
You can talk to your spouse and figure out a plan, but it's definitely not an easy road to travel. And sometimes, you can only prepare for travelling that road a few years hence.
And it sure as heck would be nice to be spared glib "just do it" advice from people who are not in that situation.
(Also, many people who are in a relationship do follow one of their dreams - being with a person they love. Sometimes, you have to choose between different dreams)