The NSA’s machine learning algorithm may be killing thousands of innocent people

Re:The NSA’s machine learning algorithm may be killing thous... by qeabc on Tuesday February 16, 2016 @07:20AM (#11109354)

Perhaps someone who knows the topic can say whether this mathematical phenomena of "adversarial examples" for neural networks can be translated to random forests: http://www.kdnuggets.com/2015/07/deep-learning-adversarial-e...

Barely perceptible changes to an image cause it to be misclassified by neural networks which never saw the image before (this is important, because it rules out simple overfitting). As a non-expert, this suggests to me that, at least for some algorithms, the reasoning "statistical algorithm estimates 99.999% probability of guilt" implies "guilty beyond a reasonable doubt" is unreliable at best.

There is great potential here for safe, effective government if machine learning output is only ever input for humans with common sense, life experience and the ability to interact with the world.

[ Parent | on HN ]

Re:The NSA’s machine learning algorithm may be killing thous... by jndsn402 on Tuesday February 16, 2016 @10:27AM (#11110427)

With respect to adversarial examples for random forests, that link states:

   Myth: Deep learning is more vulnerable to adversarial examples than other kind of machine learning.

   Fact: So far we have been able to generate adversarial examples for every model we have tested, including simple traditional machine learning models like nearest neighbor. Deep learning with adversarial training is the most resistant technique we have studied so far.

But the larger point is that adversarial examples are just one demonstration that algorithms are still quite primitive. Nobody should be killed on the basis of a statistical algorithm alone (and to be fair, the article does not show that anyone has been).

[ Parent | on HN ]

Re:The NSA’s machine learning algorithm may be killing thous... by pps43 on Tuesday February 16, 2016 @08:40AM (#11109687)

It would be difficult to create such persuasive adversarial examples in this case. Humans can recognize images well, so it's obvious for us that those adversarial examples are nowhere near what neural net thinks they are. It would not be as obvious with a list of phone calls.

[ Parent | on HN ]