I'm quite paranoid about this, yet whenever I speak to people about it either people don't care, or already accept it's happening and inevitable.
I think part of the problem is many of us already feel we've lost the battle for privacy. Although, I'm not sure we ever seriously attempted to fight for it. Every street in cities in the UK is full CCTV cameras. The underground and buses track where you travel. Our internet is monitored and logged. This isn't a future problem that will manifest from greed and advances in AI, it's something we all accept and deal with today.
In fact, a lot of people will say to this, "if you don't do anything wrong you've got nothing to hide". They welcome it.
I've almost given up. I'm pretty much despondent at this point.
There is a booth on my way to work where they give out free Lays chips in exchange for your photo. The gimmick is that it's like a photo booth. I asked the attendant to read the privacy policy, and it's frightening, to say the least. I have no doubt that they are contracting with some AI consultant to be able to identify your face in the crowd and link it to your email address, which of course can be used to identify you freely on the Internet.
People are freely giving away their identities and their rights for a free bag of potato chips.
I always try to make sure my face is covered when I walk by it, but I can't help but despair a little when I realize that I am still surrounded by security cameras. At least one would hope that the MTA police are too busy or understaffed to do anything malicious with my face.
I think that the site is not even working anymore. Anyways, they said they were not keeping the images, and I have no reason to doubt it.
It's a pity that Microsoft is going only after the corporate market, Oracle style, and not trying to make money from consumers any more. That leaves Apple as about the only big independent player in that field. We could have benefited from the competition.
https://github.com/CellularPrivacy/Android-IMSI-Catcher-Dete...
Depends on what you mean by "sufficiently" and "encrypted"!
I'm not an expert, but my guess is these are still truly difficult, deep unsolved questions... We don't even know yet if one-way functions exist; and existing modes of analysis increasingly seem insufficient to fully account for the apparent recent successes of this last wave of 'AI' on ostensibly non-convex optimization problems! (Not to mention the debates about the possibility of fundamental physical/computational complexity limits to intelligence explosion, and so forth...)
So who knows at this point; but I think it is still an open question whether a future AI could reasonably 'break' even today's commodity encryption.
But it did drift into discussing a more offensive AI than passive monitoring done today.
Most people have no mental capacity to imagine the consequences of someone misusing their private information until either it happens to them or they see it happening in some specific scenario.
What bothers me is that everyone is being carefully conditioned to misplace the blame. Person A meticulously constructs a system designed to screw you over. They willfully neglect its security for years. Then person B comes along, pushes a button and triggers the system. These days 99% of the blame somehow goes towards person B. (Evil Hackers, or that one employee who forgot to install some update, or whatever.) Mass media is totally on-board with this kind of crap.
By the way, the same applies to AI "threat". Should we worry about AI overtaking all the world's smart toasters and simultaneously overloading the to cause mass fires? Or should we worry about the fact that those toasters somehow have processors, are Internet-enabled and run shit software?
* * *
Also, "the battle for privacy" is a bad mental model. There is no single battle you win or loose. It's a gradient. The more information you leak, the worse off you are in the long run.
Weird that those people want their poops and bedtime routines live on TV globally. Having something private is not bad and anyone who thinks so hasn't taken a second to think about it.
I think the extra dangerous thing here is propaganda along with the AI to convince everyone it is okay, and I'd say that is already happening, as your comment describes.
Aside from electing responsible people to government in order to pass regulations, what can a regular person do to change the profit incentives at scale of the companies who are building the mass surveillance systems? It seems a dire and impossible situation.
There's no need to be so fatalistic. The EU is about to enact GDPR; a huge privacy preserving peice of legislation. It seems like the large technology companies are going to give the same protections globally (verification is too much fuss). This is a huge win, and US citizens get it for 'free'.
Here are some ideas for more productive things than despairing on Hacker News:
- Vote with your neurons. Stop using services that you find objectionable, even if you enjoy them.
- Stay abreast of issues via organizations like the EFF, EPIC and the FSF in the US; ORG, Privacy International and FSFE in the UK, etc. Send them your cold hard cash.
- Speak to your representatives, and give them your angle. If they're being paid by the organizations you're worried about, tell them politely what you think of this.
- Socially, as charmingly as possible, let people know what you think about surveillance if it comes up. It's a fun one if framed correctly, as it doesn't divide down left/right lines, and no-one likes to sound authoritarian. Offer help and expertise to people who are wishing to make privacy preserving changes (e.g. move off gmail, set up a pseudonym, configure a VPN, etc.)
I agree with pretty much all of the concerns you and many others have cited, and I worry about that stuff too. On the other hand, I think about the past. How much privacy did most humans have in history?
To be clear: I'm nothing like an expert on these topics.
I'm specifically not considering the wandering nomad types. I think that the vast majority of all humans who have existed lived in fairly insular communities.
So you're a member of a small town where most adults and kids work in agriculture in order to survive. The building you live in is crowded, and not just with immediate family. You sleep in the same room as a number of other people.
What privacy do you have? Cross talk/gossip moves very quickly and is pervasive. I believe if any member of this community does anything out of the norm, every member will very quickly find out about it.
I also think that most any deviance from the norm was met with, at best, suspicion.
Are you free to leave and do your own thing? Sure, but isn't that quite risky? Maybe you could join another community...maybe. But the privacy situation remains the same.
Excluding the romantic but minority cases of hearty frontier settler, and the like, I don't think humans have enjoyed much privacy at all, ever.
I'll say it again: I'm an amateur at best in these topics.
Isn't it basic human nature to, on average, form gossipy, minimally private communities?
I'm not forgetting the enormous differences between what I just described, whether it be accurate or not, and what's happening today with technology.
I'm just thinking that, perhaps, only a small percentage of all humans in history have enjoyed anything like real privacy.
PS: I'd love to receive clarifications/corrections on this from more knowledgeable folk.
For a good person, in a sense that covers most people, having your past reputation follow you around is a benefit, the ability to wipe out reputation only benefits those with negative reputation. Furthermore, if there's a general assumption that it will follow you around, it acts as a deterrent to avoid doing immoral things, since you won't be able to easily walk away from that.
The obvious limitation to "anyone can publish any information about you, tag it to your face" is the well-established concept of libel. Publishing false harmful information is already forbidden, but they should be able to "warn the world" if the information is truthful.
The amount of data Google has and the amount of data a government will have gives an order of magnitude advantage to whoever gets to harness it.
If you have to collect info on someone "the old fashioned way" (like being siblings with them or marrying them) the power differential in the relationship is bounded by definition. To me, privacy is almost like a vector with two components—the level of privacy, plus the degree of intimacy of the relationship.
Ah, you mean like the US punishing the company that leaked one out of every two social security numbers of its citizens? Or China, a country worldly-known for the Great Firewall? Or Russia, in which you either hand out data about your customers to the government, or your business fails?
I mean, the EU's actually giving a shit about their citizens' privacy, but they are pretty lonely in that fight. My country (bordering EU) doesn't give a shit, and neither does Serbia, a country that's supposed to join the EU in the next decade (yeah, I know, wishful thinking).
Even if you live in a so-called democracy, chances are that you can make exactly 0 effect. And there are all sorts of other problems which have much higher priority. FFS, I can't run to be a president of my own country. It's in our constitution that you have to declare yourself as a member of one out of three ethnicities just to be eligible to run. The country lost three cases in the European Court of Human Rights because of this and similar fuckups in our constitution, oldest of which will be a decade old next year, and our constitution is still unchanged.
I truly do care about my own privacy and about my citizen's privacy. Yet, if I decided to advocate for certain things inside my own country to make it better, I would have at least three problems much higher in my agenda than online privacy.
The usual rebuttal I see to this mindset is "well, there's so many laws out there you're bound to accidentally incriminate yourself for something minor".
Is that the real concern here? Are people more concerned with autonomous monitoring itself, or the potential outcomes/implications that come with it? I'd love to understand this mindset. (As an aside, I've always advocated for modernizing/reforming laws so that doesn't happen, instead of blaming news tools that help law enforcement do its job.)
I think we can agree there is some amount of potential benefit of this kind of pervasive monitoring (right?). I also think we can all agree there are potential downsides as well. In an ideal world, I would think we'd take an approach that minimizes the downsides while maximizing the upsides, but if people are actually taking concern with the loss of privacy to autonomous programming (i.e. non-humans), I have no idea how to measure how "good" such a system is.
(As an additional aside, there's obviously potential for misuse by human actors, but that seems like a separate issue to tackle as well: everything has potential for misuse, and needs the proper precautions and safeguards put in place to ensure it isn't misused.)
That kind of assumes legally OK is everywhere and always the same as ethical, morally OK. Which is I guess easy to do, until life teaches you personally that there's a wide gulf between the two.
"You measure democracy by the freedom it gives its dissidents, not the freedom it gives its assimilated conformists." - Abbie Hoffman
I have yet to see much in the way of concrete harms from the supposed privacy issues plaguing online advertising. Everyone has a favorite hypothetical situation where Google dobs you in to the Chinese government, but no evidence that anything of the sort is actually happening.
This is simple and harmless, but this same information has other uses that are not so harmless. They may not be being used right now, but they may start to be used in the future
Let's say you tweet some anti-American stuff from time to time and the US government decides they want you to stop. They probably won't put a bullet in your head, but they might dig up information on you and arrest you for something totally unrelated.
This happens already. It doesn't really matter what you think of them, but people like Martin Shkreli in the US and Tommy Robinson in the UK were obviously targeted by the state because they didn't like what they were doing. They dug up information and they got them on something unrelated.
AI and tech gives governments the ability to do this to new levels. They might crash your self-driving car, or they might cause your smart-TV to catch fire while your in bed, and no one would know. We'd all carry on believing surveillance is in the interest of our safety.
I haven't heard of nation-state level repercussions but the persistent profiling has already bitten a few people in the ass, such as the case where Target's profiling of a teenaged girl and their subsequent marketing betrayed her pregnancy status to her parents.
In the presence of all the tracking that people rail against, there are so few actual harms that everyone invents an issue.
I think this is telling.
Institutions have a long life. All the data collected today, too, will have an extremely long life. I am sure German Jews thought the census records, tax returns, synagogue membership lists, parish records that identified them as Jewish were harmless as well - until they weren't.
Suggesting that, amongst numerous possible scenarios of abuse, if there is nothing happening now then it has absolutely no potential for harm is an overwhelmingly ignorant position. Not to mention, the public would have absolutely no idea of such abuses even if they were happening.
But to actually answer your question, here is mobile malware [1] found by the lookout team that used 3 different vulnerabilities to complete compromise an iPhone all through a link opened in Safari. This allowed for applications that encrypt messages in transit, but not at rest on the device, to be compromised. Foreign governments used this malware to target investigative journalist and activists.
[1] https://info.lookout.com/rs/051-ESQ-475/images/lookout-pegas...
>Political security. The use of AI to automate tasks involved in surveillance (e.g. analysing mass-collected data), persuasion (e.g. creating targeted propaganda), and deception (e.g. manipulating videos) may expand threats associated with privacy invasion and social manipulation. We also expect novel attacks that take advantage of an improved capacity to analyse human behaviors, moods, and beliefs on the basis of available data. These concerns are most significant in the context of authoritarian states, but may also undermine the ability of democracies to sustain truthful public debates.
There is no need for an evil central power targeting threats to itself, but instead we have seen/are seeing the rise pervasive blanket surveillance and classification of _all_ people in a society, even non-political actors, from many different for-profit companies. The is combined with a new type of society that is entirely dependent on corporate services for almost all functions in life (food, travel, healthcare, communications, travel, work, housing, etc). It's a recipe for disaster.
This kind of state is a _new thing_^tm and we have to be aware of it. We're the ones who are going to be oppressed by it, not just the people living under the Saddam Husseins of the world.
Could an A.I. program like you are discussing have prevented the school shooting in Florida by alerting the police of the shooter's state of mind and intentions?
If an A.I. could save kids, is there any way we would not be demanding the A.I. protector be installed on every computer and mobile device today? It would be so easy to see how we would voluntarily give up power to this A.I. protector.
With correct follow-through, AI could be a useful tool for narrowing down who is a credible threat, but I agree there's a huge risk in relying on it too heavily and punishing people for pre-crime.
>undermine the ability of democracies to sustain truthful public debates.
Is coded language for exactly what you're talking about, in the same way that dictators is coded language for geopolitical enemies.
I was disturbed and appalled.
the worst part: those people will leave the class thinking they have the whole story.
the part you mention won't ever enter their mind-- it isn't under the heading of "killing" because "killing" only happens to the groups covered in class.
Back when I was an undergrad, I took one sociology class (on a different topic) and the faculty seemed especially sensitive about that sort of manipulation. I can't find a reference at the moment, but it had something to do with the CIA consulting with academics about their activities in South America in the 70s.
This was in 1987 at UT Austin.
But this "Killing" class is where I discovered that the department was segregated into distinct "tracks". They had one for general sociological studies, and one for people who were going on to be police officers and lawyers: the "criminal justice" track. This "Killing" class was part of the criminal justice track.
Later on, I took a class in CSCL (cultural studies and comparitive literature) called "Aliens" and found myself wishing I'd discovered CSCL sooner.
A lot of people unfamiliar with Machine Learning are already showing these tendencies. I am by no means an expert in the field, only a interested amateur, but I know about the limitations and I think its important that people are taught the basics of ML. More people should view ML/AI as "a complex process minimizing an multivariable error function over a multivariable input field", instead of a "computer that thinks". This way, more of these applications of it might be questioned.
I think, the only way to stop these tendencies is to teach the fundamental basics and limitations of ML in High School. It might be possible, you don't even have to introduce tensor algebra or even vector spaces to accomplish it at this stage of education.
I don't know where you're from, but a statement like that here in the United States would surely get you an ass-kicking.
Ok - a bit of hyperbole there, but not far from it; most people would simply hear "wah-wah-wah-wah" and have no clue what you were talking about; others would fall asleep.
Could you teach high schoolers what "minimization" is, and what it means to "minimize" a "multivariable error function over a multivariable input field"? Maybe some of them, but even there I'd bet the majority won't get it, and those that do will only have it for the time it takes to pass a test, then promptly forget about it.
Honestly, there are plenty of other subjects I'd rather see taught in high school (and earlier) to kids, the greatest being critical thinking methods and how to question how you know what you know. Perhaps along with philosophy (general and "of mind"), and a few other similar subjects.
I doubt we'll ever see this any time soon, though, as such knowledge would undermine much of the status-quo, particularly religion.
>> Could you teach high schoolers what "minimization" is, and what it means to "minimize" a "multivariable error function over a multivariable input field"?
By the way I phrased it in this particular way, because it is mathematically correct but also takes the magic and notions of consciousness out of the topic.
Definitely not a lawyer but, as I understand it, depending on jurisdiction and context, some social media postings may be considered "public" information volunteered without "reasonable expectation of privacy" ---in which case, awfully enough, anything goes...
Again, not a lawyer; but I wonder if there should be a right to clear-language, mandatory warnings ---like in cigarrettes--- whenever you are about to post something that will not enjoy "reasonable expectation of privacy" (and hence could be sold or used against you in the future, etc.)..
My guess is normal people don't know or care enough to turn it off.
I agree 100%
I would feel a lot more comfortable if we had an objective think tank like organization doing the research instead of the actual companies who are developing AI and have a vested financial interest in steering the public in the direction they want in order to lower people's concerns.
I've had concerns about this for a long time. Many people simply discounted me as a conspiracy buff when I brought up the dangers of AI. Now? Not so much.
The things that scare me are actors that don't have flashy names. The invisible marketing companies funded by lobbyists. Astroturfers. Russian propagandists.
privacy concerns is rather an interesting intellectual problem, and not a real one. if it was a real problem, people would change their use of those tools but they dont.
you can argue that loss of privacy allows corporations to manipulate us, but i see only trivial effects. look at how people vote and buy. i see more diversity and conclude less manipulation.
* You apply for a loan, it is refused
* Your admission, for you or your children, to a school or university is refused because more worthy people have applied
* Your job application is refused,
All because of a fuzzy social score, that is mined from your contacts, activities on social network, meta data gathered from your smartphone in an opaque way, CCTV cameras, etc
I mean if we have a way to rate people, why not use it everywhere !
This is already partially implemented in China, I remember reading an example where with a low social score you have to pay for a deposit, and you don't have to with a high social score.
In a way, that sound efficient and meritocratic, but also ruthless if you end up with a low score.
edit : add a reference with some every day life examples
For me, the dystopic part would be to have a single, automated source of "truth", like the Sesame score, used everywhere.
It means that everything becomes easy or difficult because of that score. Suddenly you cannot rent or at higher price, your insurance rate will increase, it is harder to get a job...
Optimistically, in theory, it could make people "better", but who decides what "better" or "worthy" means ?
Someone is "more worthy" according to an opaque machine learning algorithm. Your line is exactly the danger: people are going to confuse "the computer says X" with "X is true".
Where as before people would have decide that themselves even if its a shallow cursory judgement. At least people are makeing their own decisions instead of blindy following a number/evaluation with criteria they do not know.
It could be problematic if there was only one source of subjective information, especially if any perceived past transgressions were irredeemable. But if they are factual, do you have a problem with that? E.G, you go to hire somebody and find out that it's not recommended by hiring_hueristic_x based predominately on factors X,Y,Z (X being he stole from his past 10 employers, Y being he never held a job for more than 6 months, Z being he commonly posts serious threats about people he doesn't like online).
Also, do you have any idea what actually occurs when you make a decision? I don't. I would love to know someone who does. I still follow it.
It turns into a small number making decision for many. Where as without the system it is decentralized and eveyone is making independent decisions. There also more room for nuance.
Do think conpanies are going to develop there own AI and gather datasets on people just evalute potental hires. Train it for the requirements they feel is best for there company?
No they are going to out source it. So what is likely going to happen is only small number of systems making these decisons.
It is a real problem and plenty of people do change their use of their tools.
duck duck go, for example, has 0.3% market share.
you need to look at the typical john and mary on the street.
i agree that it is a real problem for lets say millions of people.
but what counts is the majority opinion for me. there are all kinds of contrary (!) opinions among small subgroups of the population, and thus it isnt possible to adjust wide-ranging policies to those opinions
Being a Jew in Germany in 1910 wasn't an issue. They had nothing to hide. In 1940 there was no going back for them.
regarding the jew example, the problem is not about having stated in public what you are but with societies which discriminate.
you might have a point saying “my life should not be transparent, as to make it more difficult for a potential discriminatory society to attack me”.
but it makes no real sense to worry about it because it is so super minor in comparison to the deep problems this society will have. i know you disagree :)
Doesn't history teach us over and over that societies DO discriminate? I don't believe that basic human trait will ever change. If that could change, maybe someday we could do away with 'evil' altogether, but I don't see that happening either.
Just look at Internet justice vigilanties. If you're on the wrong side of a social issue, you could very well be targeted. I have a few unpopular opinions, and I keep them to myself for that very reason. You don't have to look too far to see what happens if you don't.
OpenAI appears to only be thinking of the crimes-against-individuals segment of malicious AI, rather then the crimes-against-humanity type of malicious AI that the surveillance advertising corporations who are supporting OpenAI are building.
I am far, far less worried about an assassin's drones using AI to find a politician in a crowd than I am about Facebook using pictures of me that other people have posted and tagged me in, so that my face is used to track my movements, and the movements of every other human on the planet, everywhere we go, and selling that information to everybody who wants a copy, and giving it away at the request of the local police.
I'm more concerned about Google using AI to mine every conversation I've ever had or my browsing history to classify me as a dissident before I apply for a visa to travel to China or the United States, or as a deadbeat before I apply for a bank loan, or sick before I apply for insurance, or as unrehabilitatable before I apply for parole.
The hackers-on-steroids narrative is a smokescreen for fully automated corporate fascism.