Now I'm trying to learn to program. It's slow, and I wish I had started earlier. I'm too old for this shit (http://xkcd.com/447/).
A. You are not too old.
B. The problem is your attitude. You just referred to programming as "shit". Your tone implies you are referring to something you dislike.
Now, I am not trying to flame you or put you down man, but come on . . . it's your attitude that is not right, not your age.
Personally, I can't NOT program/study programming . . .
In the link there's a comic of a guy who says he can't do math like he used to, until it's revealed he's thirteen. My point was, I'm aware I could have started younger, but I didn't. That doesn't mean I'm off the hook, though. I'm still learning to hack. I agree with you that giving up on programming just because I didn't start very early on would reveal a bad attitude.
Being a nerd/geek became more socially acceptable in the last 10 years.