Actually, in Safari 4 (Intel, 64bit) each plugin runs in its own process, and it _is_ effectively sandboxed. If Flash crashes, it no longer takes down the browser, or even the tab a la Chrome; you just get a hole in the page.
One interesting test we did recently was to make a very simple flash app that draws some basic controls on the screen and scales to the window size. It also prints out the amount of memory being used by the AVM. In IE, memory usage was extremely low and was about the same regardless of the screen size. In Firefox, memory usage was higher and also almost entirely dependent on window size. To me this implies that IE is allowing Flash to interface with the video card directly, while in Firefox it's having to maintain its own frame buffer internally.
Furthermore, this argument would apply just as well to WebGL, which gives graphics hardware access directly to Javascript on any website you visit. WebGL is awesome and I believe it can be done safely.
I don't know much about QTKit but I think it's entirely plausible that it isn't flexible enough for use in Flash. Flash needs complete control over drawing because videos are integrated with other Flash content (likely rendered at least partially in software), and can be manipulated by ActionScript to produce effects such as 360 degree video: http://www.immersivemedia.com/work/cbc/hockey/index.html