I don't think everyone associates government agencies with pristine records of holding themselves accountable.
If you’ve never worked with a government agency before, let me tell you, Hanlon’s Razor is in full effect. Never attribute to malice that which is adequately explained by stupidity.
The higher ups know that free reign for their officers to look up anything is a recipe for legal nightmares. The individual officers are often just too dumb to realize that they shouldn’t search for something, and do so out of curiosity. Having blocks in the way like requiring a case number be entered to run a search, and knowing that that search and results are forever associated to that case, cut down a lot on abuse.
Are all forms of abuse the same, or so easily mitigated? No, but I’d like for some system to audit that and at least try to enforce it.
You know, my dad always told me to just listen to people... eventually they'll tell you who they truly are.
The hits were all more or less expected, a bunch around their home and the precinct, local supermarkets, etc. But then a weird grouping way out of town. Detective insisted it was a mistake and wanted more data, thinking someone stole his license plate (insert eye roll here).
Anyways, zooming in and looking at the highest density of hits, it ended up being a strip club that he frequented. He got bright red and his buddies didn't stop giving him shit for it all day.
And this was a detective, not a beat officer.
I’m not saying it’s perfect, but scapegoating the company is avoiding the real issue, which is the government policies or the enactment thereof.
The irony to me is that when I was at Palantir they talked extremely openly about all their technology. A ton of it was open sourced, and in depth tech talks were posted to YouTube, but almost no one watched them.
I think people just like writing them off as shady or sketchy because it’s easier than needing to reason about the clear ethical nuance that exists in the government contractor space.
If so, please don’t tell me you’re a DBA.
A crucial step in shooting someone (or a drone strike) is figuring out who to target and where they are.
The fact that this is an important use case for Palantir makes it not hyperbole to talk about "weaponising information".
Everything has a domain it operates in, so I believe that statement has nuance. In particular when you look at how general or specific the domain is.
From the perspective of something uploaded to S3? It sure is like any other file.
From the perspective of a website dedicated to the dissemination of child porn? It clearly is more than "just bits" in that context.
How about from the perspective of a search engine, where that site may be indexed? Welcome to the grey area that all of these debates are rooted in. Technically it's just indexed strings or ngrams, so it is like any other type of file. But there's an argument that a search engine should "know more" than just the raw data, and should be able to understand that context somehow.
Palantir is in this grey area. The software isn't built for spying. It's built for managing and understanding vast amounts of data. Can this be used for spying? Yes. It can also be used for double blind clinical trials. Or maintaining insurance systems. Or coordinating disaster relief efforts.
It operates technologically in a very general domain, but their flexible user-defined ontology makes it very powerful at operating in more specific domains. So it's a lot less cut and dry than the dissenters make it seem, IMHO.
HN won't let me reply to that, so I'll do it here. I didn't feel the need to offer a "defense" of Palantir there since I already outlined it above and don't feel like there's value in repeating myself.
But you asked, here you go - The government is going to use the data regardless, so long as they have it. The only effective recourse is to make them collecting or storing the data illegal, but good luck with that, seeing as even when it is illegal they do it anyways (see: Snowden).
Do you think is Palantir weren't there they'd just say "Ah well, I guess that's that" and be done with it? Nope; they'd go to Lockheed, Raytheon, IBM, or another long-time contractor to build a replacement almost immediately. The technology is genuinely nothing special. It federates searches across disparate database, and stitches results together.
But for me, having worked at Palantir, I know what mechanisms are provided to attempt to mitigate abuse. So as long as that data does exist and is going to be used, I'd prefer to have it be as well controlled and audited as possible. And sure, you can argue that the government may forego auditing, or be entirely corrupt, but that doesn't seem to me to be a good reason to not use tools that at least provide that capability.
So, basically "screw rule of law, people are gonna do it anyway, so let's make money off it!"
If you want to see change that is more than just superficial, that’s where you need to make it.
If someone comes into a gun shop and tells you they want to shoot up a classroom, selling them a pistol instead of a machine gun doesn't make you immune from judgement.
Want to know how a lot of that data is generated? Microsoft excel. Arguably without excel there would be less data to act abusively with. If only Microsoft just refused to sell excel to the government.
People are happy to accept that excel is general enough that it isn’t made for that one purpose, but refuse to apply the same reasoning to Palantir; that may seem like whataboutism, but they genuinely are comparable tools if you take away the marketing and fear mongering.
As for your analogy, it’s more like you own shop that makes metalworking tools, and someone buys some. You don’t know what they are going to do with it, and (here’s where it gets opinionated) you shouldn’t need to care. You sold a tool to someone. Can the tool be used to make a gun? Sure. But it can also be used for anything else involving metal work. At some point we need to accept that the responsibility for how a tool of used needs to be placed on the person using it.
We do not hear, "Government X purchases a couple thousand Excel licenses to keep a database on people it'd like to kill", and I imagine that Microsoft does not sell Excel to Government X in response to an RFP for tooling to keep track of people it'd like to kill.
(Of course, it'd probably stick its metaphorical fingers in its ears if it did hear about Excel being used for that purpose.)
Edit: Changed "we don't" to "I don't". I don't want to claim to speak for anyone else but myself.
I'm talking about neutering the surveillance state. That's not something that demonizing individual companies is ever going to do. You need to push for that change at the government level.
Unless you're pro-surveillance, but your previous comments made it seem like the opposite.
But Palantir has been the “tech boogieman” since before I even left in 2014, so it probably always will be. It is admittedly easier to blame a smaller tech company as a scapegoat than try to address the larger governmental or organizational actors that you actually have grievances against.
Just know it won’t do anything. The people who complain online aren’t their target market and never will be. And the paranoia lends way more credence to their dressed up search engine than it really deserves on technical merit alone.
So in a way the people complaining about it are making the issue they’re so upset about worse.
And yes, of course, complaining about Palantir online isn't going to change the government policy. But if working for Palantir means that no other self-respecting software engineer will want to shake your hand, then fewer people will want to work there, and their surveillance tech will be lower quality and have more holes in it.
Or you can do both. You might as well suppose that people are critical of the mercenary company Blackwater because they want to avoid criticizing America foreign policy and military exploits. I don't think that's accurate at all, it's been my experience that people who are critical of one are very often critical of the other as well.
Similarly, it's been my experience that people critical of Palintar are also critical of the organizations and governments who use the services of Palintar.
Do you ever use uber? They work to keep their drivers legally declared as contractors, despite treating them employees in all other regards. That’s pretty immoral, so I hope you don’t encourage it by supplying them with business.
Machine Learning/AI adds an entirely new perspective on big data. It can see patterns in data that would be meaningless without it, correlations that can't be found in other ways. Some data is OK for a government to have if they can just look it up in exceptional cases, but not if they act on every single bit of data collected.
In essence, using ML is creating new data (or at least: Information from the raw data). This information can be very revealing and not at all in line with the scope the data was initially collected under, which is one of the principles of GDPR.
You can't really view data as static once it's collected. Combining with other existing data and new methods of data analysis make it a totally different picture in terms of privacy.
If you don’t like the data that organizations record, I’m right there with you, but demonizing the tools that allow them to access data more effectively is an absurd stance to take. Their system also does things like add an immutable audit log, or enforce requirements on how data can be accessed.
Working with law enforcement, before Palantir there was nothing preventing an officer from looking up their ex husband or wife, or celebrities, or anyone at random. With it, there’s an audit trail of every search, and most of them need a case number as justification. Even then, certain searches would automatically get flagged for review.
We can go back to giving people direct and unaudited SQL access if you’d like, but I’d prefer some more accountability, even if the downside is organizations being able to use data _they already have_ more effectively.