For non-stream media (iso,dvds), skip files can work. These files allow you to define frames or seconds to skip or mute. For example mplayer has edl (edit decision list). http://www.mplayerhq.hu/DOCS/HTML/en/edl.html

Crowdsourcing such metadata would be a good start, although it can be subjective.

If you find out a better way for streaming media, please share.