It sounds a little odd to me that people are objecting to enforcement of the law in "neighborhoods of color". Yes, enforcement of drug possession is probably out of whack, but this isn't about that. We passed those laws, are we saying that the laws are bad, or that the laws are good, but "people of color" don't deserve to live under them? Do they not deserve to live in neighborhoods without broken windows and graffiti? Do they not deserve to not worry about getting robbed on the way home from work? Do they not deserve to live and work in buildings are properly maintained?

It seems like it's kind of a trap. If you agree with the frame of the article, then you have to conclude that either 1. We passed those laws to make our neighborhoods better, but minorities and the lower classes don't deserve that, so we should ignore some set of laws there, and that totally won't become apartheid or 2. Maybe all of these nanny-state laws made by wannabe social engineers aren't such a good idea after all, since they give the Government too much arbitrary power to be enforced on whoever they don't like this week, so we should repeal them and tell people to mind their own business when they come up with this stuff.

Or 3, you want the laws applied equally in all places and equally to all people.

The problem isn't the law. The problem is the selective enforcement. It's the selective suspicion. It's the selective police state.

Do you honestly believe that some cop in white suburbia is going to care about loose cigarettes? No way.

Do you honestly believe that anybody is selling loose cigarettes in suburbia? I wouldn't say that it's never happened, but I'd bet it's rare enough that even if cops made it a top priority, they'd have a tough time finding anybody doing it.

But that doesn't really matter anyways. I thought we were talking about not throwing people in jail and giving them criminal records for petty crimes. If a black guy gets in trouble for, say, selling loose cigarettes, is he supposed to feel better knowing that if a white guy in a different neighborhood did the same thing, that guy would be getting in trouble too? Is that really the best we can do here? I'd rather try and find a way to not throw him in jail in the first place.

I knew white kids that sold loose cigarettes in school. Depends on where you look I guess.

You're just upset you got called on a false dichotomy.

Would "75% of enforcement done in neighborhoods of color" still be an issue if 75% of crime occurs in those same neighborhoods?

This argument is so seductive, but it bugs me that people assume that "crime rates" are based on how often people are committing crimes. It's how often crimes are reported / how often criminals are caught.

Given the greater prevalence of things like stop and frisk in neighborhoods with large minority populations, is it any surprise more people are caught for things like drug possession? That's going to further skew the stats, leading to more enforcement in those neighborhoods (since they are "high crime").

Drug use is actually higher in young white populations than it is among young black populations, but because of where law enforcement spends their time the incarceration rates differ wildly.

Possession is a red herring. The CompuStat statistics don't even count drug crimes and gun possession crimes--the incidence of which might be affected by the intensity of policing due to stop-and-frisk: http://www.nyc.gov/html/nypd/downloads/pdf/crime_statistics/....

Here is the NYC crime map: https://maps.nyc.gov/crime. Look at the maps for felony assault or rape. Stop-and-frisk isn't going to change the incidence rate of those crimes.

You obviously have a more sophisticated take on the question. Parent comment referenced "75% of crime," not "75% of violent crime" or "75% of rapes and aggravated assaults recorded in compustat."

The general point holds - there are corrective measures that need to be taken to avoid skewing crime data as a result of increased enforcement, and in the case of some jurisdictions those measures are being taken.

What's not obvious to me is whether police are increasing the severity of the charges based on where they are, even if the charges wouldn't necessarily hold up in court. There's a case to be made that that would be an efficient tactic - public defenders will encourage plea bargains and it gives the DA more leverage to settle the case quickly and efficiently. The opposite may be true when booking a drunk banker who gets in a fistfight, or a privileged college kid who rapes his date behind a dumpster.

Also, the discussion generally is not about CompuStat, it's about "quality of life" improvements prosecuted with the use of secret databases that are not publicly available. So, you know, there's that.

It seems like you're making an argument that crime statistics should not be used as a metric to focus law enforcement efforts.

What other metrics should we use? If you simply assign patrol routes based on population you are going to under-serve areas with higher crime and over-police areas that don't need it.

I'm arguing that it's a more complicated statistical problem than "we see more criminal 'events' in this neighborhood, so we should send more cops there."

Offering criticism without offering an alternative isn't helpful.

Of course it's complicated, and of course crime stats are a simplification. Most stats are. The point is, stats are much better than going by biased "gut feelings".

Yes there are underlying issues with stats such as underreporting, but the solution isn't to get rid of the stats. It's to improve the underlying cause of the bias in the data, such as improving police/public relations.

Your conclusion and the one I implied in my comment are not mutually exclusive.

My solution, implied in my comment, was not to use naive statistical models or ideas (i.e. 75% of "crime").

An "implied" solution isn't a solution because it was never suggested.

And your now stated solution is easier said than done, and certainly not immune to the same kind of biases a naive solution may be subject to.

Look at the distribution of violent crime.

Yes, that would be one of many possible corrective measures.

The disparity doesn't go away if you use victimization surveys.

It's important to keep in mind that in a "racist society" machine learning will also tend to be racist (but much more so). After all, the "AI" is learning from human data, so it's certainly not as objective as you would think a "machine" to be.

https://www.propublica.org/article/machine-bias-risk-assessm...

https://enterprisersproject.com/article/2016/9/beware-biases...

http://www.nytimes.com/2016/06/23/us/backlash-in-wisconsin-a...

And similarly, in a society in which biological differences exist between races, an objective, unbiased AI will be tend to be racist.

This isn't nuanced, it doesn't add much to the discussion. It's "I think people of color commit more crimes" wrapped in an acceptable veneer.

And what if "people of color commit more crimes" is statistically true?

Since pigmentation doesn't cause crime, we should find the root cause.

Get out of here with your common sense that doesn't fit the narrative.

Is it acceptable to assign more police officers to neighborhoods of root cause?

What an enlightening comment.

I assume you are being sarcastic, but there are plausible hypothesis that could be tested to answer this question. For example the genes that have caused a reduction in human pigmentation in some populations have occurred very recently (within historical times) and under very strong selection. Genes that influence the propensity for impulsiveness or violence may have been carried along with the selection for low pigmentation.

Another possible hypothesis is that since melanin and dopamine have an overlapping biochemical pathway that the genes influencing melanin production may also influence dopamine levels.

A third possible hypothesis is that since different populations have historically lived at different densities this may have influenced how well people interact peacefully at high population densities (i.e. how domesticated people are in the same way cows are much calmer than wild cattle).

All of these are testable hypothesis that no sane scientist will ever study - in the end we just don't know the answer and most likely never will.

It is a question of a crime taking place vs one which results in an arrest, convictions and statistics. I worked on Wall St for many years -- there are a lot of drugs on Wall St, more than i've seen anywhere else. Yet the rules are rarely enforced as they are in other neighborhoods. So of course, Wall St wont make it into the statistics. It is a vicious cycle.

But how can one figure that out?

Isn't enforcement of a law (through an arrest for example) the first indication that a crime might have occurred?

If so, how to figure out if the 75% attribution is fair or just a result of selection/confirmation bias?

There are numerous reasonable proxies; locations of murders, reported assaults or robberies, calls to the police department reporting a possible crime, etc.

Is that what is actually being used, though?

I suspect there is no way for the public to know.

In-Q-Tel is a VC firm that makes loads of tech investments (https://en.wikipedia.org/wiki/In-Q-Tel#Investments) and I'm not aware of evidence that they play an active role in the company.

MongoDB is In-Q-Tel backed - so by that sensationalized metric - anyone who runs a MEAN stack is thus "CIA Backed" as well. Hilarious to see the kneejerky lack of intellectual rigor around HN these days.

Almost as hilarious as your strawman argument. Palantir was started with the explicit mission of aiding the surveillance-industrial complex. Mongo Inc was started with explicit mission of selling cloud databases(PaaS) to the general public.

Source: http://fortune.com/2014/09/04/peter-thiels-contrarian-strate...

“Basically,” Thiel explains, “I thought that some of the approaches that PayPal had used to fight fraud”—which at one point posed an existential threat to PayPal—“could be extended into other contexts, like fighting terrorism.”

For contrast: Mongo Inc's Eliot Horowitz: https://medium.com/s-c-a-l-e/mongodb-co-creator-explains-why...

What argument? You mean *[argument rest of internet always makes re: Palantir]?

Guess we found out why MongoDB had insecure defaults

tinfoil hat

It's also an interesting business model. Core platform tech and then consultancy.

A LOT of hand-cranking integrations and massaging data, presumably that's where they make most of their money - billing hours to agencies with deep pockets.

It can't be that easy. Not for a million a month.

Even the most bass-ackwards database designs can be reverse engineered by bright people in a day or two (we know Palantir can attract these people) especially if they have some scars dealing with dumb database design. And it's probably not likely that the person who built that database and the program that uses it went to great lengths to obfuscate anything. I'll even go a step further and say that most data sources are probably really simple programs. Throw in any kind of documentation as a bonus and the learning curve just goes down.

What the hell could they be charging that much for?

Mixed feelings for me, too. I'd really love to see that "all things city knows about this location" tablet app. It's just amazing to have this information easily available. But then the pesky issues of privacy and human conflict pop up, and we can't have nice things :/.

Palantir likes to promote these minimal links to the CIA quietly to increase the feeling that they give to their "forward deployed engineers" that they are some Seal Team 6 of developers. Their office in the Meatpacking district of NYC is kinda reminiscent to a cult headquarters. Rooftop bar/DJ booth unlimited* vacation time, bring your pets to work, free gym membership right nearby, every desk has a "go bag"... Why would anyone want to leave?

I've visited the NYC office for dinner a few times. There's no DJ booth, nor a rooftop bar, nor free gym memberships (according to a friend), nor go bags.

There's a rooftop patio, as many buildings in the area have, and gym subsidies as most tech companies have. I've been told they have emergency readiness kits (which are advocated by every level of government in every jurisdiction), but they're definitely not at the desks (or they are, but invisible).

How are any of these criticisms distinct from Google, which owns an entire street block (no joke street to street, Ave to Ave)? I've had more than a few meals there as well and it's pretty much the same situation as far as I can tell.