Why can't the startup be based in a smaller city and have people working remotely or if needed just the engineering team where the talent is? It all depends on the product/market.

This might be another reason, most companies probably have a tiny Coast office and say it's headquarters but have operations in another place -- to keep costs low.