There's a timing element to it too. Because CRTs display pixels from top to bottom the timing gives information about the exact location of the hit too, not just a boolean hit/miss.