News for hackers. Stuff that matters.
Re:Stealing Ur Feelings by jerf on Wednesday October 23, 2019 @09:37PM (#21340859)
Yes, it sounds like that because it was a human making up a failure mode, so it was human. Now suppose you bought 3 12-packs of Coke Zero last week, you sneezed on the frame the computer used to identify your face, the moon was waxing gibbous (which nobody even realized the AI was actually using as part of its calculations), your name is a 3 letter first name followed by a 15 letter last name, and a coincidental confluence of 25 other equally pointless things that, technically, the AI weights at non-zero. Now you're a terrorist.
Re:Stealing Ur Feelings by taneq on Wednesday October 23, 2019 @09:46PM (#21340903)
That's a way better example. Neural nets are really good at picking up obscure correlations. The last letter of your first name is 'j', there are limestone blocks in the picture and you're wearing a yellow t-shirt? Terrorist for sure.
Re:Stealing Ur Feelings by antisemiotic on Thursday October 24, 2019 @03:44AM (#21342517)
Every educated AI knows that the best way to spot a criminal is too look for people staring directly at the camera, with a white rectangle below their face.
Re:Stealing Ur Feelings by smallnamespace on Thursday October 24, 2019 @01:57AM (#21342012)
To play devil's advocate though, sometimes the ML training process is picking up a signal that humans simply haven't noticed.

The moon waxing gibbous is a fun example - what if moon phase were to affect our biology in subtle ways?

There are plenty of things that people feel or know to be true but have trouble putting into words. That guy you know who just gives you a creepy vibe, but you can't put your finger on exactly why?

Re:Stealing Ur Feelings by jerf on Thursday October 24, 2019 @02:16AM (#21342109)
To some extent, I tend to agree. Sometimes we humans are a bit too attached to our stories, and when a computer shows us they may not be true, our instinct is to simply disregard the computer, rightly or wrongly. After all, it isn't as if we haven't all had computers be wrong several thousand times for ourselves, right?

But there is also an extent to which if my example did occur, it would still be flawed, at least with modern AI. Even if all these signals are in some abstract sense truly signals that you are a terrorist, in the real world it is still not correct to simply add them together, or whatever other simple operation the AI is doing. For all that deep learning may be cool, it's still got some giant, gaping holes in it compared to however human cognition works. Humans can look at say "Yes, even if those are all 1% signals, it really isn't sensible to add them all together." The current state of AI is not able to come to that conclusion, or at least, not well enough.

If that changes in the future, well, I'll update my beliefs as appropriate then. It's hard to guess the "cognitive shape" the biases of the next breakthrough in AI may have.

Re:Stealing Ur Feelings by meheleventyone on Thursday October 24, 2019 @03:27AM (#21342450)
The problem is that correlation does not necessarily imply causation. So if a ML process finds a counter-intuitive correlation we should seek to understand why rather than assume the results are actually meaningful. Sadly I think this is the step that is being gratuitously skipped to the extent that “working” models are hardly interrogated.