That phrase only refers to the distance from you to a grocery store. So, no you don’t need to enter food desert territory when you buy a house in one of the many smaller CA cities/towns.

Anyway, buying a house can be an investment but living in a larger one than you need is wasting a productive asset. If you have a 4 bedroom house you can rent out for 4k/month then living in it is costing you 4k/month the same way renting a place for 4k would be. There’s nothing inherently wrong with buying an investment house, but you shouldn’t be out anywhere close to the full monthly cost.