Not sure about MIT and Harvard, but for Stanford CS courses, if you go to csXXX.stanford.edu you'll typically get the class page, including slides, problem sets, programming assignments etc. Ex. http://cs221.stanford.edu.
Obviously, I agree with you. In fact, I'd go a step further and say that watching the lectures without doing the problems/coding gets you significantly less than halfway.
MIT has a whole directory of openly available stuff at http://ocw.mit.edu/OcwWeb/web/courses/courses/index.htm
It will be really great if someone made a forum/website where people could interact with each other about a particular course and ask questions and have discussions regarding certain topics.