You don't need the value to remain in any one place. You just need your house to not tank relative to the rest of them so that you can leverage up when you get rich and develop a fetish for "good school district" and downsize when the next is empty.

It's a store of value, not an investment. The goal isn't to get rich. It's to not have to pay rent in retirement (yes, I know property taxes).