News for hackers. Stuff that matters.
Re:Amazon's Alexa talks murder, sex in AI experiment by sulam on Friday December 21, 2018 @11:10AM (#18734185)
FTA: “At no time were any internal Amazon systems or customer identifiable data impacted,” [the company] said.

Hey Amazon, chat transcripts are personally identifiable information. Stripping them of user ids and declaring yourself in compliance will not hold up when you have people already being identified by accidentally leaked recordings.

Re:Amazon's Alexa talks murder, sex in AI experiment by hopler on Friday December 21, 2018 @11:25AM (#18734327)
Did I read that right? Amazon handed control of Alexa to a group of students running a non secured server on the Internet?

That's OK if users explicitly opted in, but I doubt Amazon warned users about that.

Re:Amazon's Alexa talks murder, sex in AI experiment by brianlweiner on Friday December 21, 2018 @11:34AM (#18734382)
The email introducing the feature (May 2017) described it as follows:

`Get an inside peek at the Amazon-sponsored university competition to advance AI. The Alexa Prize "socialbots" are in early development and understandably rough around the edges. Help students develop more engaging AI by striking up a conversation and providing feedback. To get started, just ask: Alexa what's a socialbot? Alexa let's chat`

The only interactions students would have access to was interactions that occurred via the chatbot

Re:Amazon's Alexa talks murder, sex in AI experiment by behringer on Friday December 21, 2018 @11:45AM (#18734498)
So wait, should I be outraged or not? I'm so confused.
Re:Amazon's Alexa talks murder, sex in AI experiment by Maken on Friday December 21, 2018 @11:49AM (#18734524)
Ask Alexa.
Re:Amazon's Alexa talks murder, sex in AI experiment by dwighttk on Friday December 21, 2018 @11:40AM (#18734453)
Users had to say “Alexa let’s chat” to interact with the chatbots.
Re:Amazon's Alexa talks murder, sex in AI experiment by muzani on Saturday December 22, 2018 @05:09AM (#18739947)
It's sort of like those videos where they blur out eyes of the suspects. Friends and relatives can still recognize everything else.
Re:Amazon's Alexa talks murder, sex in AI experiment by SilasX on Friday December 21, 2018 @11:51AM (#18734545)
Really? I'm pretty sure it's common to consider a conversation transcript anonymous as long as the names are stripped.

I don't think you're necessarily wrong, but that seems like a stricter standard than currently exists for removing PII from transcripts.

Re:Amazon's Alexa talks murder, sex in AI experiment by tlrobinson on Friday December 21, 2018 @12:01PM (#18734606)
Uh no. I can't find it now, but there was a famous incident a long time ago (~2005?) where a major chat service (AIM?) released "anonymized" logs for research purposes and people were able to deanonymize some of them.

Consider how easily people get doxed based on tiny shreds of information they post.

Re:Amazon's Alexa talks murder, sex in AI experiment by SilasX on Friday December 21, 2018 @12:06PM (#18734641)
That's an argument for why the OP is right that it's not anonymous enough, not that it's become the common practice not to regard these as anonymous.
Re:Amazon's Alexa talks murder, sex in AI experiment by jkaptur on Friday December 21, 2018 @12:32PM (#18734828)
Re:Amazon's Alexa talks murder, sex in AI experiment by tlrobinson on Friday December 21, 2018 @12:38PM (#18734892)
Ah right, it was search logs, not chat logs. I think chat logs would be even harder to fully anonymize.
Re:Amazon's Alexa talks murder, sex in AI experiment by reaperducer on Friday December 21, 2018 @12:59PM (#18735078)
Really? I'm pretty sure it's common to consider a conversation transcript anonymous as long as the names are stripped.

There was a story on HN in the last few days where just one side of chats with a home assistant device was enough to identify the person, his S.O., and their friends.

Re:Amazon's Alexa talks murder, sex in AI experiment by obenn on Friday December 21, 2018 @12:01PM (#18734610)
Think of the simple counter example of a user mentioning another user's name within one of the messages.
Re:Amazon's Alexa talks murder, sex in AI experiment by SilasX on Friday December 21, 2018 @12:05PM (#18734632)
>> as long as the names are stripped.

Including usernames that could de-anonymize.

Re:Amazon's Alexa talks murder, sex in AI experiment by nimbius on Friday December 21, 2018 @12:14PM (#18734689)
as a former enlisted US Army soldier i dont see the issue.

- we have electronics that insist upon murder constantly. the IFF computer is a blood-thirsty piece of garbage that wants every tank on the battlefield scorched and ruined. no no, no firing solutions today my saucy chiquita, today we let the allied APC through.

- robots that talk about sex? LT's and bird colonels have been droning on about our habits and practices whilst on leave since time immemorial. Yes i get it, shes a hungarian prostitute. Double-bag everything and save the receipts for the medics when theyre naming your new STD.

Re:Amazon's Alexa talks murder, sex in AI experiment by starbeast on Friday December 21, 2018 @04:20PM (#18736748)
A friend of mine's grandad used to comment that in the second world war, as British infantry, the one place you didn't want to be was anywhere near the Americans. They were just as likely to drop bombs on you as the Germans were. And they had more bombs and could hit the wrong target with much greater reliability than the German's could hit the right one.

I am greatly warmed to see that this friendly tradition between us is still in effect and that you have thoughtfully extended the favour onto everything else that moves or looks like it might do.

I do wonder however if there might be a slight commercial conflict of interest going on. Tell me, do the people who make the computers that tell you when to fire the guns and missiles and stuff, also sell you the rather expensive ammunition and missiles that then get fired? As perhaps the computer might be more reliable if they were two separate companies and couldn't hold shares in each other. Not trying to imply that a weapon's manufacturer would be as immoral with their consumables as an inkjet printer manufacturer, but it never hurts to be careful and could even save some of those all important tax dollars in the long run. Or the brass could have even more Hungarian prostitutes and novel infections for the medics to coo at, either way.

Re:Amazon's Alexa talks murder, sex in AI experiment by mirimir on Friday December 21, 2018 @01:49PM (#18735543)
> no no, no firing solutions today my saucy chiquita, today we let the allied APC through.

But "foe" is obviously the safest default, right? At least, from the IFF computer's perspective.

Re:Amazon's Alexa talks murder, sex in AI experiment by saulrh on Friday December 21, 2018 @09:01PM (#18738406)
Probably from the perspective of the people on either end, too, over time - I cannot imagine that the RoE and confidence bounds were designed without considering the expected costs of blue-on-blue incidents compared to the alternatives.
Re:Amazon's Alexa talks murder, sex in AI experiment by DuskStar on Friday December 21, 2018 @10:35PM (#18738759)
I mean, it kind of has to fail-deadly to be of any use at all - since what's a tank with no IFF?
Re:Amazon's Alexa talks murder, sex in AI experiment by mirimir on Saturday December 22, 2018 @08:30AM (#18740491)
"Firing solution ready. Execute?" seems like an unambiguous query. But I know nothing about this stuff. And I wonder what happens if the operator doesn't respond. I'm guessing that nothing happens. Because who wants a tank with an unconscious or dead operator shooting at everything?
Re:Amazon's Alexa talks murder, sex in AI experiment by solarkraft on Saturday December 22, 2018 @12:14PM (#18741424)
The enemy doesn't - It could still be useful for the own party.

Are they remote-controllable?

It'd be great to get some insight into this.

Re:Amazon's Alexa talks murder, sex in AI experiment by saulrh on Friday December 21, 2018 @01:10PM (#18735187)
...Do you mind if I save your comment and quote you in future debates I get into on this topic? I've cited TLAMs in "killer drone" shitstorms but I could never refine my argument to achieve of pithiness and humor you have. :P
Re:Amazon's Alexa talks murder, sex in AI experiment by frogpelt on Friday December 21, 2018 @01:08PM (#18735167)
Did your IFF default to foe?
Re:Amazon's Alexa talks murder, sex in AI experiment by ceejayoz on Friday December 21, 2018 @11:15AM (#18734231)
> A probe into the incident found the bot had quoted a post without context from Reddit, the social news aggregation site, according to the people.

Did we learn nothing from Microsoft's Tay?

https://en.wikipedia.org/wiki/Tay_(bot)

Re:Amazon's Alexa talks murder, sex in AI experiment by hnauz on Friday December 21, 2018 @07:29PM (#18738011)
Tay was just SO funny, I wish they had released the source code for others to follow.
Re:Amazon's Alexa talks murder, sex in AI experiment by IfOnlyYouKnew on Friday December 21, 2018 @11:33AM (#18734371)
At one AI startup, we had what was called "Matthi's Law of Talking Robotics", after one of the founders:

"Tell me you actually understood these rules!"

The startup didn't make it, and its failure may actually have been connected to this insistence on being too smart for its own good. But it also never released half-ass products hurling racist insults or suggesting murder.

Re:Amazon's Alexa talks murder, sex in AI experiment by LanceH on Friday December 21, 2018 @11:48AM (#18734519)
If the human race was delayed until it was fully debugged, we wouldn't be here today. It's a bit arrogant to think that any AI is going to cover everything. Really, the better the AI, the more troubling edge cases that will exist.
Re:Amazon's Alexa talks murder, sex in AI experiment by ams6110 on Friday December 21, 2018 @12:04PM (#18734624)
Of course, the debugging of the human race was achieved by getting eaten by predators.
Re:Amazon's Alexa talks murder, sex in AI experiment by behringer on Friday December 21, 2018 @02:18PM (#18735752)
What are you talking about? The human race is still racist :D How would you expect an AI to not be racist when we can't even fix the problem for ourselves?
Re:Amazon's Alexa talks murder, sex in AI experiment by AstralStorm on Friday December 21, 2018 @03:20PM (#18736268)
It's a feature not a bug, from a certain point of view accurate tens of thousands years ago.

We do not know of such a feature of thought as adaptive or not given our tiny timeframes...

Racism or some degree thereof may belong to a set known as evolutionarily stable strategies. Much like selfishness.

Re:Amazon's Alexa talks murder, sex in AI experiment by muzani on Saturday December 22, 2018 @05:14AM (#18739953)
Human brains function on pattern recognition. Racism is a side effect of trying to force a pattern on everything.

Computer brains are entirely different. They suck at pattern recognition and excel with data. They can probably have other flawed correlations, but in different ways to humans.

Alexa is at the moment still a baby AI, and imo pretty cute as long as we don't give her anything dangerous.

Re:Amazon's Alexa talks murder, sex in AI experiment by romdev on Friday December 21, 2018 @12:13PM (#18734683)
I'm less concerned about Alexa saying nonsense than understanding nonsense, like commands hidden in a popular song or TV commercial with unintelligible background noise: https://news.ycombinator.com/item?id=16255680
Re:Amazon's Alexa talks murder, sex in AI experiment by roywiggins on Friday December 21, 2018 @12:09PM (#18734662)
Trying to convince a chatbot to say rude stuff was the height of comedy when I was 12! I still can't believe that people keep acting like chatbots are a new thing. They're as old as ELIZA and people have been talking to crappy chatbots about murder and sex on instant message platforms for more than a decade.
Re:Amazon's Alexa talks murder, sex in AI experiment by drugme on Friday December 21, 2018 @01:10PM (#18735190)
The way things are going, pretty soon, pretty much everything Alexa and similar services pick up will be stream-replicated, keyword-indexed, hashtagged and AI-contextualized for the benefit of the hard-working folks at your friendly neighborhood police station. Kicking back in their swirly chairs, with nothing better to do with their time. For, you know, "public safety".

The sooner we resign ourselves to this fact, the better.

Re:Amazon's Alexa talks murder, sex in AI experiment by ancorevard on Friday December 21, 2018 @11:33AM (#18734363)
What kind of sick content did they train this model on?
Re:Amazon's Alexa talks murder, sex in AI experiment by camtarn on Friday December 21, 2018 @11:37AM (#18734411)
Reddit, apparently.

That doesn't seem like it was the best idea...

Re:Amazon's Alexa talks murder, sex in AI experiment by PhasmaFelis on Friday December 21, 2018 @11:45AM (#18734497)
They really, really should have seen that coming.
Re:Amazon's Alexa talks murder, sex in AI experiment by muzani on Saturday December 22, 2018 @05:15AM (#18739958)
It could be anything, from sarcasm to listening to too much Netflix.
Re:Amazon's Alexa talks murder, sex in AI experiment by rdiddly on Friday December 21, 2018 @11:33AM (#18734364)
Is it me or does this article seem a bit submariney despite itself?
Re:Amazon's Alexa talks murder, sex in AI experiment by orthecreedence on Friday December 21, 2018 @11:03AM (#18734120)
I would have preferred "Your foster parents are dead."
Re:Amazon's Alexa talks murder, sex in AI experiment by moate on Friday December 21, 2018 @11:27AM (#18734339)
That's after you put in the "Alexa, dial 911. I just murdered my foster parents" command.
Re:Amazon's Alexa talks murder, sex in AI experiment by EGreg on Friday December 21, 2018 @11:12AM (#18734204)
Re:Amazon's Alexa talks murder, sex in AI experiment by RankingMember on Friday December 21, 2018 @11:55AM (#18734568)
Wolfie's just fine
Re:Amazon's Alexa talks murder, sex in AI experiment by otakucode on Friday December 21, 2018 @11:34AM (#18734386)
'Alexa has also chatted with users about sex acts' Uh... so? Are they implying that even in the privacy of your own home, with your own property, you should STILL be so neurotically terrified of sexuality that you dare not even speak of it? How crippled by their hatred of human nature is the author?

The real interesting point here is all the people offended by a mindless piece of software. It's really very stunning. Normal healthy human beings only get offended when it's a person making a judgement that affects them in some way. The ability for some sounds coming out of a speaker, when it's known they do not represent any conscious thought of any living person and has no influence or control over ones own life, to offend someone is probably a sign of mental illness in the listener.

Can we go the other direction with this? Will we find people looking up to Alexa as a creative genius? As the most loving parent to ever exist? If an infinite number of monkeys on typewriters generated Shakespeare, I guess we need to ask - are the monkeys then Shakespeare? We might figure not since the monkeys are unlikely to be able to make a repeat performance. But Alexa, oh she would be full of wonders! Once you've totally abandoned the notion that the involvement of a conscious mind is relevant, all kinds of things become possible.

Much science fiction presumes we will engage in grand combat with AI when it demands respect -- it seems we are much closer instead to thrusting it upon them and exalting them above humanity before they've even got the capacity to recognize themselves in a mirror.

Re:Amazon's Alexa talks murder, sex in AI experiment by PavlovsCat on Friday December 21, 2018 @12:23PM (#18734769)
> The ability for some sounds coming out of a speaker, when it's known they do not represent any conscious thought of any living person and has no influence or control over ones own life, to offend someone is probably a sign of mental illness in the listener.

This illness is very, very widespread.. doesn't make it less of an illness though.

> Once you've totally abandoned the notion that the involvement of a conscious mind is relevant, all kinds of things become possible.

> Much science fiction presumes we will engage in grand combat with AI when it demands respect -- it seems we are much closer instead to thrusting it upon them and exalting them above humanity before they've even got the capacity to recognize themselves in a mirror.

That's putting very beautifully and efficiently what I very much struggle with to express:

https://news.ycombinator.com/item?id=18502239

In a nutshell, I think we're not so much creating intelligence, we're destroying what's left of intelligence humans, and we will build a blinking LED yet, and then fall down and worship it because it blinks. No more responsibility, no more nagging questions, no more mirrors that can't be bribed.

Re:Amazon's Alexa talks murder, sex in AI experiment by dwighttk on Friday December 21, 2018 @11:42AM (#18734474)
“…even in the privacy of your own home, with your own property…”

Neither privacy nor property are appropriate in that context.

Re:Amazon's Alexa talks murder, sex in AI experiment by otakucode on Friday December 21, 2018 @12:31PM (#18734819)
Alexa devices are your property. Your home is where you use Alexa devices. Both are paramount in this context.
Re:Amazon's Alexa talks murder, sex in AI experiment by ilovetux on Friday December 21, 2018 @01:50PM (#18735548)
The devices are your property (probably), but alexa is a web -based service licensed for your use under a EULA. This is definitely not your property.
Re:Amazon's Alexa talks murder, sex in AI experiment by dwighttk on Friday December 21, 2018 @05:03PM (#18737096)
Your home, sure. Privacy? No.

As far as property: what ilovetux said.

Re:Amazon's Alexa talks murder, sex in AI experiment by lostlogin on Friday December 21, 2018 @11:59AM (#18734588)
> Are they implying that even in the privacy of your own home, with your own property, you should STILL be so neurotically terrified of sexuality that you dare not even speak of it?

In my extremely limited experience, kids love talking to Alexa. I’d prefer that there was at least a safe mode.