Nearly every person I've met struggles with empathizing with people who aren't in their in-group, that's not a homeschooled-specific thing. The enormous polarization in the modern political climate is due to precisely the kind of insularism that you're calling out, and it's not because of the 1.5% of the adult population that was homeschooled.