Good point about screen sizes in general, but perhaps you miss the main appeal of small portable devices (aside from aesthetics alone): portability!
And it matters too for portable screens: otherwise 5" palm pilots, extra-large tablets, or 19" laptops would have been the norm, don't you think?
Whether it's for functionality or for status, bigger is better for a lot of things. Phones were the odd duck because the screens were tiny back then.