I get that too. I ask what the idea is, and then find some innocent-sounding reason to point out why I wouldn't be a good person to make it, rather than saying I'm not interested or I don't have the time. If the app exists, I'll tell them about it.

Either way, people are happy that they had the chance to talk about their idea and I'm not stuck doing something that I don't want to do.