E.g. The Fourth Amendment warrant requirement. No one doubts that we'd find more criminals and contraband if police could make impromptu home searches of anyone at any time. But that's how a police state functions.
And we've decided to exchange efficiency in crime reduction for civil rights.
Assuming they are already doing this, wouldn't it be preferable if they switch to a model that decreases the number of innocent people that are searched by 10x? Wouldn't that be a net good to society?
Of course if your opinion is that they shouldn't be allowed to do random searches at all, this whole thread is really irrelevant.
(You'd have to find a different metric to figure out what effect the 4th amendment or similar equivalent in other countries has.)
Many signals have absolutely nothing to do with wrongdoing whatsoever. Say that hypothetically every drug dealer makes large cash deposits, and this is marked in a database somewhere as a "very reliable indicator". Well what about taxi drivers? Street Vendors? other occupations that deal almost exclusively in cash, should they also be suspected?
What happens if somebody otherwise innocent of anything in particular triggers multiple "flags" like this, do they deserve to be discriminated against?
What if the flags are stuff like being black, being poor, having a friend that has been charged with a crime. What about living in the "wrong" neighbourhood, how about the wrong genetic markers. How about a family history of mental illness, what about if you dont own a cellphone? What if they are angry about the government and have publicly protested? What if you accidentally like ISIS on facebook, or your friend does it as a joke? are these acceptable "black flags" too?
What about if one of your close friends trips one or more of these markers? What if several do? Are you just as untrustworthy?
It gets much choppier when you introduce something like neural net learning into the equation. It will form links between activity that are basically impossible to decipher by a human at all, should we accept these utterly opaque judgments because "on average they are pretty reliable"?, what about the cases where they are 100% dead wrong, should they just be considered collateral damage?
A huge class of people could end up going through life denied basic services like a bank account because of things ENTIRELY beyond their control, or even understanding, all they know is they managed to piss off some equation somewhere.
It gets better, because on average people with bad scores on one system "reliably indicate" they will be bad faith in other contexts too...
Lots of businesses will accept this as an acceptable cost to keep fraud rates/whatever down, I don't find it acceptable, its almost the definition of injustice, and has a grave potential to create a vast underclass of "poor social credit score" people (who being a member of, even by association will assuredly be another hit against your scores)
How happy would you be, if via no fault of your own you were deemed an "unacceptable risk"?
So yes, it is inhuman, and wrong. I guess that -100 Social Credits for me.
Where profiling becomes dangerous is when it's tied to something you are or something you did in the past: it can be harder to identify why you're being profiled in these cases. Most of these cases aren't resolved through a single text, either. Profiling is a useful tool, but we have to be careful to do it in a humane way.
How is this any different from any number of discriminatory practices in the past, which are now (at least superficially) illegal, such as "red-lining" black neighborhoods to deny home loans?
‘We Kill People Based on Metadata’ [1] http://www.nybooks.com/daily/2014/05/10/we-kill-people-based...
Example - http://www.acamstoday.org/domestic-high-risk-geographies-eme...
Jesus, dude. Obviously they should.
Who the fuck cares about efficiency if they're pulling over people because they're black... in sedans.
Probably quite a lot of drugs are moved by white guys in shiny rented cars, because those are less likely to be profiled.
easy. ignoring protected classes [1] - thereby losing part, but not all of your signal. i.e. don't see race.
Should the police instead do less efficient completely random searches? Should the taxpayers hire more police officers to make up for this loss in efficiency?
Should businesses accept losses from fraud to placate your feelings?
While it may not feel very good to be sending signals that associate you with crime, that is a cost you have to accept of sending those signals.
In the case of police searches, I think attention would be better spent on making them more humane, rather than less racist.