epiceyebags
Either way, you can start at any age when learning anything, especially when it's something that is primarily self-taught.
The career aspect is hard to answer. It's probably going to be harder to get an entry level job at a large publisher if you're not a fresh early 20s graduate. Then again, the industry has changed to the point where being a codemonkey at such a company isn't as coveted as it once was. It's all about small, independent companies now and you'd reasonably expect less age barriers.
Look at the engines people are using at the moment, and pick something popular. Unity is the big thing right now and definitely worth looking at from an employability standpoint going forward in the next few years as tons of small start-ups as well as established independent studios are using it. It's built on C++ and uses C#, Javascript and Boo (a Python-esque language) as scripting languages.
Also, as someone with a game development degree, my advice about going to a college is this: don't. It's a waste of time. The one exception being if you're in the US and can get in to DigiPen, as students from there seem to consistently produce interesting, high-quality stuff and do well in the industry. I have no idea if their game-centric courses are focused on programming or not though.