> Wouldn't any physical structure that's not homogeneous and isotropic also break symmetry in the same way. Does the room I'm in break symmetry as there's a different likelihood of hitting a wall depending on the direction I travel?

Uhuh, but enough to be physically detectable?