I started when I was 10, but I didn't get seriously into programming (like deciding it was going to be a profession) until I was 23. Also had a heavy math focus, though it was more because I enjoyed math than because I knew I wanted to do something that needed math.

There's plenty of time to try things out in high school and college, and it doesn't make sense to specialize until you have a sense of what's out there and what you might enjoy. The world is much, much broader than it seems in high school.

i didn't start until i was 22... i'm 24 now. am i doomed?

You learn by doing. So keep doing it.