These aren't incompatible. Household sizes are falling and number of homes is an input to population, so you can have residents moving out on net while prices increase due to the remaining residents being more wealthy. California is an entire state of ~40 million people where this happens: low and middle income families move out; high income families move in.