Thing is, the definition of "small" is changing. My 4.6" Sony Z3 Compact is considered "small"!
Do the majority of smartphone users have large hands? I have fairly average-sized hands, and my Z3C is nearing the limit of what I can comfortably use one-handed.
Phablets may not be for everyone, but neither are the small phones that are rapidly becoming an endangered species.