I actually did that once, for some badware on "download minecraft".

All but one of the badware ads I reported disappeared, within 20 minutes too! (but why one remained I have no idea!)