For precision shooting, the alignment of the sights with each other has a much bigger impact on where you hit than the alignment of the sights with the target. An intuitive explanation for why might be that the distance from either sight to the target is much greater than the distance between the sights, so a misalignment between the sights creates a much larger angle than the same amount of misalignment with the target.