It isn't about "idiots" or stupid "manufactured outrage"
This is HN; if you are going to invest your time in creating a business the last thing you want to do is blow your market opportunity by associating your brand with something as dumb as penis pics and nazis.
Is it manufactured outrage that Microsoft paid dancers in schoolgirl outfits at GDC? Maybe, but it's irrelevant.
Don't mess up your brand. I have no idea what Microsoft is doing right now. It's like a company going through the equivalent of mid-life crisis trying desperately to be "cool".
It just seems kind of obvious that Twitter users would grief the bot. Now, they address this point, but it's interesting that they still didn't last a day.
Think about that -- they were expecting abuse, but they still lasted less than 24 hours. That's certainly interesting.
People try to social engineer people all the time, and it works. The consequences are somewhat limited because you are limited to breadth (mass media, phishing etc.) or depth (one on one interactions), but can still be scary.
If an AI isn't more resistant, we face the risk that any re-purposing of data from one AI in other copies, or allowing the AI to massively multitask makes social-engineering of AI far more wide-reaching, and hence far more worthwhile for attackers.
It'll be interesting to see what kinds of attacks will get directed at e.g. customer service bots in the future, and what brands ends up damaged as a result.
Hopefully this gets integrated right into Cortana. "Cortana, how do I get to the hardware store?" "Turn left and <unspeakable sex act from Urban Dictionary> yourself"
Well, if those "idiots" include potential investors or customers of future MSFT AI products, "don't worry about it" is not sound advice. This is a very public failure of a promising Microsoft product.
I mock the way they communicated the incident because it sounds too sci-fi. Almost too Ghost-in-the-Shell like. But I do not mock the technical effort in any way.
This is just as stupid as that manufactured outrage over Googles image tagger. It misclassified a picture of a human as an animal, and people were up in arms. Google had to censor it so it can't tag animals now. They shouldn't have to do that, let idiots be idiots.