I suppose that because people buy or vote for what they're told to buy or vote for, this will let those wanting to influence people more effectively spend their money. But the solution to that is not AdBlock, it's education. Smarter people mean less susceptibility to manipulation, which is what we are really trying to achieve, right?
It depends on who buys the data and / or the interpretation of that data from them. Just imagine a country like China buying it to predict who will become a subversive and arresting them before anything happens, something akin to pre-cog crime. When I think about it, we can probably aleady do this somewhat accurately with all the data we can collect right now. We have all the tools: open (and affordable proprietary) ML software, open big data frameworks (hadoop, storm, actor model, cassandra), as well as the cloud (AWS, Rackspace). People just need a comprehensive set of data.
> Smarter people mean less susceptibility to manipulation, which is what we are really trying to achieve, right?
Not necessarily, you can't constantly consciously fight thousands of years of evolution; but that's another topic.
Then there's the argument that in 30 years the US will be one of these countries, too. I wonder why people are so afraid of this, but not of the small chance that they'll be hit by a meteor when they go outside tomorrow to go to work. They've been hit by a meteor exactly as many times as their government has used a "social graph" to oppress them, after all.
You're vastly underestimating the potential value of access and storage of more and more precise behavioral data combined with the innovation of a startup vs the stodgy, uncreative, stubborn ways of both old world telecom and finance corporations. Back then, even with access to people's purchasing data and past addresses, I would have had to do a lot more work and ask for larger increases to our budget to figure out people's relationships as well as non-purchasing behavioral data aside from traffic. Now I can potentially have access to people's preferences to stuff as opposed to just guessing. Today it's much easier and cheaper.
Right now machine learning is still a niche area for the majority of programmers. Looking at open source software landscape for ml, this is slowly changing. It's only a matter of time before people make breakthrough applications (that is if they haven't already).