I'm not sure "bad" is fair. But the app ecosystem wasn't really developed, the network connectivity wasn't great, and there were probably a lot of other shortcomings especially in retrospect. I had a Treo at the time and didn't upgrade for a few years to the 3GS which, as I recall, was when the iPhone really took off.

The iPod had a somewhat similar trajectory. The first gen version was pretty much just another MP3 player and iTunes didn't even run on Windows at first.