Give it a try. The lectures I looked at don't look like they require much more than being able to read basic assembly language, and some of them probably don't even need that.

The GPU lecture is particularly good and doesn't seem to require much more than basic programming knowledge.