I did the math once and realized that the cost of a modest apartment large enough for my family in SF/SV would exceed my total cost of living in Southern California.

SoCal is not exactly cheap. That's completely and utterly insane.