If you're borrowing $970K, you're monthly payment would be a lot more than $4K. When I was looking at mortgages, borrowing $600K was ~$4500/month in mortgage, property taxes and insurance. $970K would be more like $6K per month.
The only program that does do that, FHA, have loan limits and in SF for a single family home that is about 625k...
BUT, in hot markets like SF and the east coast, no seller is going to accept your offer with only 3% down because of the risk the lender having an LTV issue and the sale falling apart. In markets like NYC, Boston, DC, SF, etc you need to come in with a) offer over ask b) at least 20% down.
Here's a better plan: Buy a huge house for a fraction of that money in the north of England, rural Spain, anywhere in Japan which isn't Tokyo, etc., and telecommute to work.