It took me about five years to save my down payment. I lived very frugally, don't have a wife, don't have a kid, I paid less than $20k on note for each of my cars, and do not have college debt. I'd say it's fairly difficult if you start moving any of the knobs I minimized or set to zero above. Even then, after my down payment money was tight for the first year because it was everything I had.

I got a mortgage similar to what OP mentioned but I have a veterans home loan. Most people I know don't have one of those, so their mortgage that was gotten at the same time as mine is closer to $4k/m.