We understand that when two galaxies hit each other, the visible mass gets all tangled up and suns and nebula smash into each other. Meanwhile the invisible mass continues onwards, oblivious to everything else. We CAN observe it and we have.

Seriously this is basic object permanence. We can see hidden things moving around by themselves. Dark matter is still there even when there's normal matter on top of it.