I have a flag on my platform that flags users who reported things I agreed with the removal of, as "trusted" and their reports are given more weight in the algorithm next time. Sort of the inverse of what you propose where everyone is somewhat untrusted by default. (But not entirely untrusted.)