It's because philips drivers are wedge-shaped. They properly fit a range of screw heads. This means that there's only two or three commonly used driver sizes. With torx the sides are parallel so the driver size is directly related to the screw head size.