The networks themselves rely almost exclusively on javascript nowadays so the websites have little choice, the ad networks then in turn pass some or all of this trust to whoever makes the creatives, which up until recently were quite frequently done in flash and are now sometimes in javascript.

Personally I think all ads should be served up in a totally passive visual format (png, jpeg, gif) and have no other attributes than a non-javascript link target. That would take care of almost all drive-by injection. But adnetworks serve up what their customers want and their customers want interactive ads because the click-through rates are higher and because otherwise the competition would be doing it and they go out of business.

Ad networks that do serve up javascript should at a minimum pull the script to their own server and audit the code of the script. Good luck with that though.

Fortunately it's easy enough to install an ad blocker and get rid of that part of the problem entirely but it would be nice if users without an ad blocker wouldn't have to worry about this.