After watching through most of the series, and writing a 3d engine (HH style, no libraries), I have internalized an amazing amount; the vast majority of what Casey's gone over. I agree that often times he goes into more detail than necessary - even for me, who started knowing literally nothing about what he was talking about, but I would attribute the amount of learning I've gotten out of the series to exactly that. For someone that doesn't understand the difference between an Object Transform and a Camera Transform, the only way for them to actually learn what they are is to hammer it home again and again - through examples and practice. I think I am exactly his target audience - young programmers who want to learn how to make video games.
Just my 2c
Game development is a great way to either brush up on your linear algebra skillz or motivate you to develop them in the first place, so he can't assume that his audience is either fully up to speed or completely ignorant.
I know from my own experience, though, that it can be "fun" to explain things that you know inside out in (too much) detail because it gives you a feeling of satisfaction and appreciation of the material you're going over.
However, I can't help that for anyone to whom you have to explain high school math episode after episode, the rest of the show, i.e., actual (and sometimes rather low-level) programming would be completely over their head.
That said, I do enjoy watching the shows quite a bit and I still plan on finishing the full run. I just wish the trivial stuff wouldn't receive so much time, or that there was a good way of skipping forward without having to fear to miss something important.
One really awesome thing is that the source code is bundled up after each episode so that you can really easily get back to speed after a longer break, like in my case. What I plan to do is to find the last episode I watched, then download that episode's code, and remind myself of the current state of affairs by studying the code, rather than trying to remember the content of everything I already saw.
I think this whole series is such a cool thing to do. And you can really tell that he knows his shit.
Casey, if you hang out here on HN, two thumbs up for this awesome effort!