I've always upgraded my iPhone only for the camera and I've never regretted that. There are so many photos I have from a long time ago that I wish were higher quality, so I'm always willing to spend $ to make sure that today's photos are as good as they can be. 90% of the time I don't have my proper camera with me, so the phone is what it is.

Tangentially, after seeing some NFL replay highlights this weekend, I really wish HD high-frame rate cameras had been invented when Barry Sanders (perhaps the most electrifying football player of all time) played.