News for hackers. Stuff that matters.
Re:Cameras That Can Spot Shoplifters Even Before They Steal? by simonh on Wednesday April 24, 2019 @11:05AM (#19739071)
The thing with machine learning systems is that we can't always know what specific characteristics or behaviours they have learned to respond to. All we know is that they have associated some characteristics of shoppers as indicating a high likelihood of imminently engaging in theft, but we might not be able to un-pick what those characteristics or behaviours are from the neural net weightings.

I think a potential risk here is from small initial sample sizes. Here's a thought experiment. A system like this is set up and runs successfully for some time. A person with some distinctive characteristic enters the store and steals something and is caught. The system learns that so far 100% of people with that characteristic stole. Next time it detects that characteristic, it flags that person and they're ejected from the store. The system will never learn that this characteristic can be benign, because from now on it will be excluded from it's data set.

That's a bit of an artificial example, making assumptions about how the system might work, but something along those lines could be a potential failure mode, and we might never know what that characteristic is because the system can't express the reasons for it's decision in a meaningful way.

There are no comments on this comment yet.