Google collects immense of data about people's actual visits. Backlinks used to be a proxy for how authoritative things were You don't need the proxy when you have the record of where people actually visit.

That's very effective for the top results. But if legit sites cannot rank in the first spots they'll get orders of magnitude fewer visits than the scam sites that employ SEO hacks to get the first places.