I don't have iPhone 4, and can't say much how is the performance. But according to my experience, it's software issue (memory management). After upgrading my 3GS to iOS, my daily task is to kill all apps in the multitasking bar when it feel sluggish. I can check from iMemGraph, the free memory is increased substantially.