Computer yechnology work is still a wild west. Other fields which have similar ethical considerations - medicine, chemistry, engineering - have ethics courses which make it clear to the graduates that their professional decisions have consequences.
There is no such standard for computer people. It still baffles me that people would voluntarily work at implementing something like PRISM - I would literally quit my job over something like this; similar considerations have popped up in the research I do for my Masters thesis - but ethics is definitely something we would talk more about.
I don't know whether you're aware of this or not, but you just described a famous experiment which essentially arrived at the same conclusion. It's worth a read for those not familiar with it.
Zimbardo has stated that he is still pained at the memories of the Stanford prison study, which had to be aborted prematurely because the vile characteristics it exposed in normal, healthy participants was going out of control. He reluctantly revisited the study in detail because he saw striking parallels to the torture and humiliation in the Abu Ghraib prison under US occupation. While he's at it he also reviews much other research into the psychology of authoritarianism, obedience, dehumanization and other facets of social psychology that add up to what may very reasonably be called Evil.
I might add that organizations like the CIA are well aware of these principles, and appears to have been working on ways of weaponizing them. It is my opinion that they are using them to corrupt individuals and whole social environments when that suits their ends. In fact, in spite of "The Lucifer Effect" being a fascinating read from beginning to end, one of the parts I found most interesting was the evidence indicating that US intelligence actively created the psychological conditions for the abuses at Abu Ghraib.
Highly recommended reading, and very relevant for much of the potential consequences of mass surveillance.
This interpretation isn't really any less disturbing, since it suggests that people will commit atrocities and simultaneously believe that they are doing the right thing. Perhaps it also says something about the general public's trust of, not necessarily authority figures, but rather scientists.
Reading up on this has been on my back-burner for a while, this interpretation might be well considered and discarded, but I thought I'd throw it out there.
I believe this is related: http://www.is.wayne.edu/MNISSANI/PAGEPUB/Milgram.htm
Perhaps the difference is if the doctors/nurses/vendors think that they can get away with it, not attracting the public's attention.
Now if we could get some prominent figures in tech with some authority to get more involved in these matters, but no - they're all a bit too comfortable in their well-paid, US-based jobs to risk anything.
PRISM people thought they were doing right. All NSA - based on all the official and unofficial responses - think they are doing right. That they help building a safer America. That they have to sacrifice privacy for safety. Etc, etc.
I don't think NSA people are evil. They believe that the things they do are for good.
Evil like in the movies doesn't exist. In the real-world evil is simply a lack of perspective (some would call it empathy instead).
No one ever wakes up and decides they want to be a villain. They always have some sort of logic that rationalizes their actions as being reasonable if not outright good. The more they act on that lack of perspective the greater the evil they perpetrate.
Some select quotes (minor spoiler alert):
"There is no conspiracy. Nobody is in charge. It's a headless blunder operating under the illusion of a master plan."
"It's all the same machine, right? The Pentagon, multinational corporations, the police. If you do one little job, you build a widget in Saskatoon, and the next thing you know, it's two miles under the desert, the essential component of a death machine."
Quentin: But why put people in it?
Worth: Because it's here. You have to use it, or you admit that it's pointless.
Quentin: But it is pointless.
Worth: Quentin... that's my point.
DIN_FIV
Escape To Reality
Track #3 - "Conspiracy"
http://www.youtube.com/watch?v=X2ktuDmEAv8I worked with a sociopath in the past and have studied it in literature, and a sociopath by definition sees meeting their own needs as a priority. Regardless of the effect on those around them.
A sociopath is willing to inflict suffering on others for even minor gains of their own. Sociopaths seek out high-power positions and often thrive in them, and those working with them often suffer as a consequence.
Taking a step back, as far as I can see, a sociopath is someone who (either by choice or by nature) prioritises certain social drives over other social drives. The drives to be empathic and obey social expectations, norms and rules get ignored. However, the drives for status, money and power are prioritised above all others. These drives are still social in nature. They don't actually convey a fundamental biological advantage.
I personally choose to aim to be a warm and caring human being, and, as a consequence of that, I have a really great relationship with my girlfriend. When we have kids, our kids will grow up in a loving supportive environment and so will have a good chance of growing up strong and well balanced. Being a sociopath would probably get me more material possessions, but I would have had to settle for an emotionally weaker partner (who I could dominate) and I would end up with messed up kids with a lower chance of success and survival.
From my own personal experience, people who fit the sociopathic archetype aren't really like evil villains. They're more like computer game addicts, fixated on goals that don't bring them happiness, and that get in the way of forming genuine connections with other human beings. I can see why people who are the victims of their behaviour characterise it as selfish, because they see the world as a competition for money, status and power and they think they are losing out. However, that competition is just a game, and the grand prize is not happiness.
Although I find these points interesting, it does seem to me like this leads us into the age-old philosophy discussion on ethics. It seems like ethics can be argued to be based on intention, effect, something else, or all. I have read about sociopaths that are experienced by others as good people. However, a sociopath seem to have a stunted emotional life and I am sure this disability will always have numerous subtle negative effects on people that have an emotional relationship with them (e.g wife, kids, friends).
This is what I consider evil, and I think it's important to separate it from people who think what they are doing is ok. I.e. "Someone else would have done it anyways", a thief who steals from a bank because "they have insurance" or "they are rich assholes who don't deserve it", or a dictator who tries to do what he thinks is best for the country even though his policies are bad.
Tou keep saying that, but it doesn't mean what you think it means.
Evil in this case is an aggregation of actions by many parties, not individuals and is rarely committed by the tool makers. However, it's always justified by the perpetrators.
Yep. Of course if that engineer had perspective he would have to wonder if there was more to the story than just what he was being shown. It is easy to doubt the people we already think are wrong, the hard thing is to doubt the people we agree with.
I would add that I think much of the material Ed Snowden released, to the extent that it shows personal expressions, to me points more to these people viewing the Intelligence organization as the new "we", and /everybody/ outside, US citizen or not, as a bit less than fully human. That cheeky smiley on the sketches detailing how they broke Google's SSL? "TOR Stinks" and similar flippant expressions in the documents about attacking the integrity of TOR? And so on. To me that isn't the look of someone who thinks they're making difficult, serious choices to protect the greater good. It's the look of someone who thinks they're better than everyone else out there and they can do whatever they want, including having "a little fun" toying with those other inferior creatures, because they think no one can touch them. That, I think, is the general mindset we're dealing with. Allow for a fair deal of individual variation of course.
You don't need an ethical guidebook to consider the implications of the work you're doing. I don't think lack of explicit instructions is an excuse. I don't want to invoke Godwin just yet...but suffice to say, a lot of bad things have been done in the name of good. And there were people in the loop who had the ability to see what was going on.
And so they walk a famously paved road.
ACM Code of Ethics: http://www.acm.org/about/code-of-ethics
So, the standards exist but people choose to ignore them. Which, I guess since you're not forced to be a member of either professional society as a "computer person", isn't all that unreasonable. But personal ethics should win out in these situations without someone else having to tell you that it's wrong.
1.7 Respect the privacy of others. ..
User data observed during the normal duties of system operation and maintenance must be treated with strictest confidentiality, except in cases where it is evidence for the violation of law, organizational regulations, or this Code. In these cases, the nature or contents of that information must be disclosed only to proper authorities.
That specifically allows the kind of work done by PRISM.
2.7 Improve public understanding of computing and its consequences.
Computing professionals have a responsibility to share technical knowledge with the public by encouraging understanding of computing, including the impacts of computer systems and their limitations. This imperative implies an obligation to counter any false views related to computing.
3.1 Articulate social responsibilities of members of an organizational unit and encourage full acceptance of those responsibilities.
Because organizations of all kinds have impacts on the public, they must accept responsibilities to society. Organizational procedures and attitudes oriented toward quality and the welfare of society will reduce harm to members of the public, thereby serving public interest and fulfilling social responsibility. Therefore,organizational leaders must encourage full participation in meeting social responsibilities as well as quality performance.
Or number one on IEEE?
1. to accept responsibility in making decisions consistent with the safety, health, and welfare of the public, and to disclose promptly factors that might endanger the public or the environment;
Given, what's "best for the public" is up for debate (at least according to the people doing the collecting).
That does not make our decisions inconsequential. And yes, we should talk a lot more about ethics in compsci courses.
As for people who work on things like PRISM, it often happen in small steps, one ethical shade of gray at a time. When you get to a PRISM, your worldview is so completely distorted, you think it's a natural thing to do.
It's almost like not everyone in the world shares your political views on privacy rights...
Seriously, you are baffled by this? The majority of the population doesn't value privacy at all.
Thank you. I was going to make the same point, you've done it well.
Moreover: this is why the Snowden, PRISM, NSA/FBI, surveillance, privacy, and other stories matter on HN.
Many of those working for startups are contributing directly (and occasionally indirectly) to the datastreams which are being harvested. I've never felt comfortable with Facebook and the ethics of its leadership, and I've been increasingly concerned about Google (I consider it to have lost rather than sold its soul). Both are far too actively carrying the NSA's water (as I've pointedly told Google multiple times).
How to answer the challenge ... I really don't know, I'm at a loss here.
Maybe I am missing something, but I don't see where your dissatisfaction is coming from.
By creating an infrastructure in which the data are collected, centralized, individually identified, unencrypted, and specifically associated with vast trails of other nominally unassociated online information (search history, logins to other sites, analytics on other sites, etc.).
This is in the sense of Lawrence Lessig's Code and Other Laws of Cyberspace. The architectures of Facebook, Google, and other social networks, advertising networks, mobile phone systems, phone apps, PC-based snooping (inclusive of Microsoft Windows, OS X, Linux, Android, and other systems), etc., etc.
These systems are designed in ways which maximize their utility to a national surveillance network, despite alternative architectures being available. As an example of alternatives, Twitter and Reddit, at the very least, don't demand / promote "real names", or (mostly) track information across third-party sites, at least not to the extent Facebook and Google are doing.
"Treadstone! Ahh yes, Treadstone. Treadstone was a pilot project to <blah blah blah>. It's all but decomissioned at this point."
What they provide is an f.lux experience, with a +1 magnification. their optics are quite clear, but no where near the marketing hype they claim.
When I compared a $10 pair of sunglasses to a pair of +$100 Oakley's I was like wowzered by the optics. It actually provided me with better contrast across the range, in all but rainy weather.
Always wanted to try one out after i saw that movie. So one day I got a tax benefit and could get them for free. They shipped out the wrong model first time, so I got two.
They seemed really overpriced to me, but do help. Keep them on long enough and you'll see blue. I had clear benefits when for a year long I was doing online poker zombie shifts. But on one without a frame, the legs came of fairly easily. They were plastic and glued into the glasses. I still have to re-glue them, but never bothered. I might fix them and give them another go.
But for $100+ I'm getting a solid yet lightweight frame from Oakley, and a crappy plastic frame holding not-so-impressive optics together.
Other differences that come to mind: Google has yet to kill people with drones or send people to torture prisons in Egypt.
It's a big government.
"Just doing my job" has been deprecated in 1945. For a reason, and finding out why the hard way is considered harmful.
Even in 21st century America, we continue to fool ourselves, thinking that somehow soldiers can be noble, heroes even, all the while being engaged something that is immoral.
With an all-volunteer army though? No way, it is on them as much as it is on anybody. I don't care if they joined for the GI Bill (doing it for the [college] money is a pretty fucking flimsy ethical justification), I don't care if they joined the reserves before war broke out for the GI Bill (agreeing to the possibility of a war, before you know the circumstances of that war, for the [college] money, is still a flimsy argument).
The "support the troops [bring them home]" stuff seems like it is likely backlash against the derision that we gave Vietnam veterans, but it should be re-evaluated because the situations are not equivalent.
I was going to detail this as a possible excuse - but considering this reasoning and the consequences has made me reconsider: You've made a bad choice [morally / ethically]. Now you have to live with the consequences.
Can be seen as a tough stance, but then again, so's life. Didn't really mean to fall of that cliff? Oh, here's your life back again.
Still, I don't hate people who join or anything... call it a very strong disapprove. However I do find it hard to associate with people who try to assure me that they're one of the good ones because "Oh, I'm not really the patriotic type, I just joined so I could go to college" line. That impresses me far less than "I joined [for my country|it was the right thing to do|for the innocent civilians over there|war is a necessary evil|I was young and naive when the recruiter came to my school|911 HUURAH!]". I don't believe people when they say the former, but if I did, I don't think I could consider them anything but mercenaries.
Here is a combat veteran discussing what it is actually like. This sentiment is basically echoed among the friends that I have who have served in the military.
http://www.mediumdifficulty.com/2012/03/01/call-of-apathy-vi...
"No, I have told the whole truth in that, and only dead men can tell the truth in this world. It can be published after I am dead." -- Mark Twain
Noam Chomsky:
Anything that's totally vacuous and diverts, after all what does it mean to be in favor of .. suppose somebody asks, do you support the people in Iowa, can you say I support them or no I don't support them. It's not even a question it doesn't even mean anything. And that's the point of public relations slogans like support our troops is that they don't mean anything, they mean as much as whether you support the people in Iowa.
Of course there was an Issue -- the issue was do you support our policy but you don't want people to think about the issue that's the whole point of good propaganda, you want to create a slogan that nobody is gonna be against and I suppose everybody will be for because nobody knows what it means because it doesn't mean anything, but it's crucial value is it diverts your attention from a question that does mean something. Do you support our policy and that's the one you're not allowed to talk about.
http://www.chomsky.info/interviews/199201--.htm
http://fixyt.com/watch?v=G7DdWmWUa_8
Incidentally, this passage is within a larger context of public relations. A while back I was looking at terms and phrases on Google's Ngram viewer and noted something really odd: "democracy" wasn't a term in broad use until World War I, with a second major uptick during WWII.
https://books.google.com/ngrams/graph?content=democracy&year...
I noted that and didn't think much more of it for a few months until I ran across a reference to Edward Louis Berneys, father of modern public relations.
http://en.wikipedia.org/wiki/Edward_Bernays
Really interesting coot. Double nephew of Sigmund Freud (oh yeah: Berneys popularized Freud too -- and married advertising to psychology while he was at it), convinced women to smoke ("Torches of Freedom"), and though it's hard to believe it had to be sold, promoted bacon.
One of his first campaigns? A WWI "support the troops" message commissioned by the US government. To popularize democracy.
That's right: the principle American Value was brought to you as a marketing campaign.
(Ngram viewer is fascinating for studying the emergence / rise / fall in the popularity of words and phrases).
More on lies of diversion: https://plus.google.com/104092656004159577193/posts/5zrCkbzR...
The nonsense message of American democracy is another one that's always bothered me, so it's enlightening to see how these are connected.
FWIW, despite all the propaganda and even our school history books, democracy never was a core value of America. There is virtually nothing democratic about the design of the federal government, and even those aspects that are democratic were nothing new at the time the Constitution was written. America was not a bold experiment in democracy. That was old hat: the Dutch were already doing it, and heck, the Greeks did it a couple thousand years earlier. What was a bold experiment, seemingly entirely forgotten in today's discourse on Capitol Hill, was a limited government that wielded only a small set of powers that were explicitly ceded to it by the people.
I'm not sure I'd go so far as to call it "nonsense", but it definitely wasn't emphasized prior to WWI. I'd have to do some more digging through history to see what issues were driving the nation prior to that. I do think that freedom and liberty were there. Initially it was just getting things off the ground and fighting off the Brits (1776, 1812), then westward expansion, then early industrialization, then slavery, them more industrialization.
By the time of WWI there'd been some big questions raised over equity and equality (railroads, robber barrons, oil trusts), and national cohesion was somewhat at stake, more so with the Great Depression and WWII.
My study of all of this is still pretty thin, but it's a fascinating period.
If you believe that the war is unjust, and you believe that it's the duty of a soldier (or any person) to refuse to perform immoral acts, then consistency dictates that you should condemn the soldiers fighting in those wars, not revere them.
I've spoken with a lot of Iraq war veterans. Every one of them has qualms with how the war was conducted, and many have doubts about the reasons for invading. But the things they have in common are an underlying desire to to "protect" their country in some sense, and the willingness to die for that cause.
There's plenty of room there for both disagreement and respect.
I have friends who are absolutist peacenik liberals, who thing that GWB should burn in hell for all eternity for inveigling us in Iraq, because of "war for oil", "Halliburton will destroy the country", etc. From the point of view of one of these people, there is no moral authority to wage the war; to them, it IS black and white. Yet these very same people utter the "support our heroic troops" mantra.
Given that someone is such an extremist in their opposition to the war, it's impossible to square that person's professed beliefs with their support of the soldiers waging that war.
That is, unless we're willing to discard the idea that all people are morally bound to refuse to execute immoral commands.
They may not have made the decision to start a war (that is on politicians), but they made the decision to participate.
"The CIA's Silent War in Pakistan"
http://content.time.com/time/printout/0,8816,1900248,00.html
"CIA drone attacks in Pakistan are responsible for unlawful killings, some of which could amount to war crimes, Amnesty International says."
http://www.bbc.co.uk/news/world-us-canada-24618701
"CIA Drone Strike Kills Pakistan Taliban Head Before Talks"
http://www.bloomberg.com/news/2013-11-03/cia-drone-strike-ki...
The division between "intelligence" and "operations" has been breached.
Does the CIA admit to even owning drones? Or are these Army operations run by the CIA?
The CIA's drone program is not secret, and hasn't been for some time.
From http://www.nationalreview.com/corner/363199/obama-plan-shift...:
"Despite their overlapping “orbits” in Yemen, the CIA and JSOC employ different surveillance equipment on their drone fleets. They also rely on separate and sometimes incompatible communications networks to transmit video feeds and assemble intelligence from multiple streams in the moments before a strike."
From http://killerapps.foreignpolicy.com/posts/2013/11/05/cia_pen...:
"Keeping the drones with the CIA also offers legal cover for drone strikes, former officials argued. By law, the military is not supposed to conduct hostile actions outside a declared war zone, although special forces do so on occasion acting at the CIA's behest."
Some other choice headlines on the site's homepage:
Birth-Control Battle - Liberals are the agressors
Chait-Crime: Stop assuming conservatives are racist
Corporations Are People: The idea goes back to the Founding.
The Return of the Queue: O'care: just like any other liberal program.
Five Food-Stamp Myths: Does the program even work?
The Test-Score Gap: Race and class are not to blame.
Not exactly a fantastic citation, I think you'll agree. In any event, your quote specifically refers to intelligence drones, not strike drones. I'm not actually 100% sure about this, but I'm guessing they're different, just based on the different payloads required to perform each activity. In any event, the CIA here isn't admitting to anything. It's an anonymous inside source. Right. Hyperconservative blog has high-level inside source in the intelligence community in a post-Snowden world. Got it.As for the second article, that's exactly what I'm saying. Special forces do so, "Acting at the CIA's behest". CIA provides intel, special forces conducts the operation. That's an important distinction.
If you're intent on nitpicking, then I think the likely answer to your question is that the CIA openly runs a large and mature weaponized drone program using vehicles that are outfitted by the CIA but owned (in a procurement sense) by the USAF, with the weapon triggers actually pulled (per law) by USAF service members.
If the CIA wanted to keep this program secret then they're doing a shitty job. For example, look up the things that Lean Panetta has said about his program over the last few years.
http://mccollum.house.gov/press-release/mccollum-amendment-f...
http://unmannedsystemscaucus.mckeon.house.gov/in-the-news/20...
http://unmannedsystemscaucus.mckeon.house.gov/afghanistan/20...
http://www.judiciary.senate.gov/pdf/04-23-13BergenTestimony....
More to the point: there's no denial that these programs are run by the CIA.
I'm pretty sure it is...
If Google deletes your comments on a YouTube video, it's not a violation of the 1st Amendment. If Google employees take a good, long look at your email or the documents you have in Drive, it's not a violation of the Fourth.
The Constitution only limits Google to the extent that it's the basis for the system of laws that do the limiting.
Not Google nor any other person is granted rights under the Constitution. Given the Natural Rights philosophy subscribed to by many of the founders, as described by John Locke and others, the rights of the people pre-exist, simply by virtue of their being people.
The Constitution comes later. It describes a set of enumerated powers that the people have decided to cede to the government that they're creating. The way you describe it is the other way around from this.
At face value, yes. But with the news that companies are tracking web users via browser fingerprints and other methods (even when users have "do not track" enabled), it seems that it is difficult to avoid being secretly tracked by corporations.
There are many commercial services offering lookups (google "HLR lookup")
NSA Director Keith Alexander disclosed in Senate testimony in October that the NSA had run a pilot project in 2010 and 2011 to collect “samples” of U.S. cellphone location data. The data collected were never available for intelligence analysis purposes, and the project was discontinued because it had no “operational value,” he said.
Alexander allowed that a broader collection of such data “may be something that is a future requirement for the country, but it is not right now.”
And for all this, I'm not going to lie--that'd be a fun dataset to hack on.
And that, folks, is why we as engineers and hackers have a moral duty to be very selective in the types of work we take on and for whom.