Helps for both search users and the platform. Like a free moderation service. Why have an employee too check if these pages should be removed if a crowd of tracked users can do it.
I wonder where is my view wrong because I wonder why isn't there such system yet...
In order to detect malicious activity you'd have to use a bunch of other signals, and also to use the downvotes with some suspicion, so you're back at looking content quality, reputation, page speed, and many other signals. Which sounds very similar to status quo.
Personally I'd like to see some sort of explicit input. If not used collectively, at least to customize my own results, like it happens on YT/Netflix/Instagram/etc - e.g. "never show results again from this site", or "I dislike this type of content".
there are tons of ways to prevent that by weighting votes and only taking a random sample. SEO is basically a war where SEOs find a way to exploit Google's algorithm and Google tries to respond.
Backlinks were Google's original solution to keyword spam that made other search engines suck, they didn't expect people would create entire sites or pay other websites to send links to them so they could rank higher. Backlinks were themselves considered to be "votes" for quality similar to citations in academia. The problem is the barrier to entry to make a site and link to it is high, so SEOs themselves are often the only ones to do it. Adding some sort of voting mechanism would allow actual readers to give an opinion on quality, although you'd probably see SEOs pay people to vote on their link via a VPN switching IPs, so only people with established Google accounts could be allowed to vote. Obviously its a huge challenge, which is why Google is worth hundreds of billions for even partially solving it