Saving for retirement doesn’t necessarily mean 401k but you can borrow from your 401k for down payment assistance so you really should be saving for your eventual retirement from day 1.
Don't forget too that inflation works for you! If you save and get in a nice house early and the STAY THERE (I can't emphasize enough how important this is - sometimes life forces you to move, but avoid it), your payments measured against inflation will go down - just put that difference into a 401k (that is don't do a cash out refinance along the way as so many do) and in 30 years the house is paid off and your have a great retirement account.
When you are in the early years the above plan looks impossible, but time is your friend.
In all honesty, when was the last time you tried to rent a place to live? My brother makes decent money in his mid-20's and _still_ wound up living out of his car for a couple months when his attempt to find something cheaper backfired.
Plans fail. Have backup plans. Attempting to min max has disadvantages too. My advice has always come from experience and never “this is what they say you should do”. I would not be so presumptuous.