As long as you know a lil bit of math and coding, you're good to go. If you're watching the cs231n videos and don't understand something, you can always check out their amazing class notes.

The cs231n course covers a lot of material, including some (or most) of the material from the Andrew Ng's Coursera course. You can totally do them in parallel though.