That by itself is scary enough, but the much more likely case is the one where this system is rating you based on wrong information, and given Finagle's law..
Especially concerning when the justice system starts using this info; you really, really don't want to be the false positive.
They can't just say "he had a bad score, so I pulled him over and found weed". If they did that, the stop would be unjustified and the whole case would be thrown out.
In a low margin business a single bad customer can easily cost more than what is earned from ten good customers. In that situation, an oracle that rejects eight good customers per rejected bad customer would already be good enough.
If anything, they seem to be doubling down on this. From my discussions on the subject with people "in marketing", they are really convinced that not only is the data "accurate enough", but that collecting and exploiting it is actually for the client's benefit.
I wonder how one would go about quantifying the usefulness of all this tracking, because I doubt it's cheap. The best I could get was "we're able to see changes in sales which correlate to marketing campaigns". I can totally buy that and it seems somewhat more advanced of an answer than the more common one of "If didn't work they wouldn't be doing it".
What I wonder though is how they would go about quantifying how much better a tracking-based campaign worked than a "traditional" one would have.
How would they know?
But there's no process to appeal and the data might be laughably wrong. Outlawing cross-organizational aggregation might be a reasonable middle ground. It gives a further advantage to giants like Amazon, but all the problems compound at interorganizational integration.
"2. In addition to the information referred to in paragraph 1, the controller shall provide the data subject with the following information necessary to ensure fair and transparent processing in respect of the data subject:
...
(g) the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject."
But that requires that the company even cares about following the GDPR correctly, or perhaps even the Data Protection Authority of a given EU country. I know of one small but popular enough data broker that's been operating for years in the EU and flaunting the GDPR without any punishment so far. Not sure if this is the place to advertise it. But I suppose somebody whose GDPR rights were violated has to lodge a complaint first.
I had drunkenly called HSBC and ended up ranting at the person on the phone a few times (lost cards etc) so I think this was a rating of how well behaved I was towards their staff.
The manager I was speaking to changed his tone fairly quickly after that screen came up!
So this is nothing new, but I guess the scale and opportunities for data points are new.
Although, having said that, a lot of marketing based scoring data is woefully inaccurate. I was once responsible for distributing a data set to company clients which covered interests and personal info for all UK population.
Not only was most of the data about my dad wrong, the things that were accurate were years out of date.
My information didn't exist. A colleague's email was completely wrong. Indicated he liked going on holiday but had never been outside of the UK.
So these scores might be useless anyway.
edit: got rid of "even if they exist".