Some people is not enough to support massive economies of scale required for major cellphone companies. This means you eventually need to pay more for a smaller screen, and that’s really unpopular.
PS: I have average sized hands and without a case I can easily use an regular iPhone 7/8/X with one hand. Cases make this far more difficult, but imo that’s why phones seem to converge on that size.
Why is it so hard to believe that some people DO prefer smaller phones?