Well, I'd say most of the problem here is trying to identify a two-dimensional screen by a single number, the length of its diagonal. It must make more sense to give the actual dimensions of screens.
I remember pained discussion of the Nexus 5 saying the 4 was wide enough. The 5 is only wider by a couple millimeters, but is significantly enough taller that they bumped the single number identifying its size. That's no way to communicate.
It's all trade offs but as we increase what we do with our phones, a bigger screen potentially becomes more valuable.
I know a lot of people saying they have no interest in more than 3.5" or 4". Generally those people are fairly casual users. I also know "power users" (horrible term) who have been crying out for bigger screens.
I'm less worried about thumb size and more worried about how it feels in my pocket.
Part of being Steve Jobs is knowing when you have a breakthrough and knowing when you've got an incremental improvement. He did plenty of the latter as well as the former.