I did start there actually. As much as I love Khan Academy there are lots of holes and the quality/thoroughness of video explanations really seem to drop off the further along you go.
To me they were most useful when I just needed a refresher on the precise operations to go through in order to solve a particular kind of problem, but already understood it. It's hard to beat a textbook for learning material that is completely new to me.
I was able to start at the beginning and get all the necessary math to actually step forward to some of the more complex courses. I'd suggest starting there.