Between tests a student can slack off for a couple of weeks. Then spend a week catching. Going to the math center or talking to a tutor. They take the test and then repeat the cycle. Most teachers give practice tests and for the most part it's obvious which type of problems are on the test. Just need to learn those types of problems. The material covered in the class is much more than the few problem types covered on the on the test.
Obviously my particular failure doesn't indicate anything. That's why I wrote:
"My experience is...."
The flaw is that when students don't watch the video it can't be hidden in class. When doing group problems or in when discussion occur it's clear they are lost. It's cleat that the work wasn't done. It's this embarrassment - the confrontation with the reality that they didn't do the work - that is upsetting to them. It causes problems and is a great pain to deal with.
I believe that the format proposed only works with motivated students. My belief might be wrong. I don't think it is.
In fact, I did this a couple weeks ago myself for a course in which I had no particular interest and yet was forced to take as a graduate student, doing the minimum amount of studying necessary to be reasonably certain I would get an A (spending my nights and weekends hacking instead of studying archeology).
The wider problem then, I suspect, is people studying things they aren't all that interested in, which encourages procrastination and cramming. This may be a necessary feature of any sort of broad curriculum -- people should show competency in areas even if they don't have a particular interest in them. If so, then this does indicate "a flaw in the idea," insofar as the presumed idea will not work across a large spectrum of people and thus may not be implementable at the collegiate level.
That said, you could theoretically modify it to have more frequent gradable quizzes on the lectures that people are supposed to watching at home, forcing them to watch them at the assigned times instead of attempting to assimilate the information at the last minute before the test.
Your particular failure to successfully implement an idea doesn't indicate a flaw in the idea as a whole.