I've worked in education with enough people to understand this situation very well.
There's an initial "hump" that you have to get over, the very first time you are learning how to program. It involves wrapping your head around a few dozen distinct concepts and how they can interact.
For some people, it takes a few days, and for others, it takes a few weeks or months. This is usually based on how much prior exposure they've had to programming - how much they tinkered with it before trying to learn it formally.
Anyway, after that hump, people feel confident that they can learn to do anything in the world of programming. In my experience, the only people who failed to reach this point were the ones who did not dedicate a reasonable amount of time.
Once you get over those initial concepts, your skill level is really just a matter of how much experience you've had. The ones that learned the initial concepts in a few days are not necessarily better than the ones who took a few months.
People say this a lot. That is the entirety of the evidence for it. I have yet to meet someone in a context in which they actually wanted to program who didn't have the ability. Programming isn't hard, hard programming is hard, because it usually requires hard math and deep abstraction. In those cases, though, the programming is the easiest part - it's pencil to notebook that's frustrating.
"I haven't seen the evidence" is hardly synonymous with "there isn't any": http://www.eis.mdx.ac.uk/research/PhDArea/saeed/paper1.pdf