I myself remember "sweating blood" to get definite integration; unfortunately never got a chance to do multiple dimensions.
Or take organic chemistry. I have a particular type of memory inherited from my paternal grandmother, anything I can fit into an organization of related things I retain well. And I have really really good 3D visualization. First term organic chemistry was easy, and I didn't have to use physical models more complicated than my fingers and knuckles. It's well known that at minimum this is a lot harder for a lot of people. Heck, it's an appropriate weed out class for premeds, appropriate in that it tests both of those capabilities which will later be needed to learn medicine.
By the time a person is old enough to be taking math classes, their "innate" abilities are a combination of nature and nurture. Hitting a "breakpoint" might be the result of deficiencies in their nurture, rather than truly innate (i.e. genetically determined).
The question is, if you took every person and raised them in the optimal way, how far could they go in math, programming, or any other pursuit? I think it's obvious that there would still be differentiation according to talent, but I also think it's likely that the average would be a lot higher than it is.
I totally believe in innate talent. There are standard breakpoints in math teaching where people simply don't get it. No matter how much time you spend explaining it to them, their brains aren't wired up to handle certain levels of abstraction. At best they remember it briefly, but it never sticks.
(The breakpoints are roughly at basic 'manipulate equations to find x' algebra, basic calculus, and calculus in many dimensions.)
Being good at software and math requires an innate ability for symbolic abstraction and reasoning. It also needs a good memory, so you can keep a lot of details in your brain at the same time.
If you don't have those, you're not going to be able to do the job - certainly not well, and possibly at all.
Hard work will make the most of natural aptitude, but there has to be some natural aptitude to start with.
If this isn't obvious, I'd suggest learning some completely new skill, like painting or music. If you don't have natural ability you'll spend a lot of time flailing around wondering how the hell anyone can do it at all.
That's how most of the population feels about coding. The HN population is a self-selecting group that feels otherwise - but that doesn't make it typical.