Nope. That would be too slow. I tried that a couple years ago before writing Seriously.js, and the big problem was copying around the memory. You have to first draw the video to a canvas, then you have to extract the pixel array from the canvas, loop through all the pixels and then draw the pixels back to the canvas. Now, for a high-definition video at 4 bytes per pixel (RGBA), you're looking at about 3.5MB of memory for each copy. And on top of that, it has to allocate and garbage collect that 3.5M pixel array for every frame, because it doesn't let you copy into an existing array. If you look at the first video-processing canvas demos, they tend to be pretty small, and this is why.

Instead, this uses WebGL/OpenGL. So the pixels are copied once from the video directly into the GPU memory, and they are processed in parallel there before going straight to your screen. Much, MUCH faster.