It's a possibility, but it could be complex to put in place (client and server side). You have to find a way to aggregate the data in a smart way, to avoid getting your system messed with and minimize false positives.

Just imagine the case where a site changes drastically, all the crowdsourced results would be out of date, how do you detect that ? Because a user signaled it ? What if it has been incorreclty signaled ? Sure you could set a sort of treshold, but that means that a bunch of users will have a poor user experience in the meantime.