News for hackers. Stuff that matters.
Re:Cloud Video Intelligence API by imh on Wednesday March 08, 2017 @04:07PM (#13823798)
The demo picture they chose is interesting. It's obviously a tiger, and is identified as such with only 90% probability. I appreciate the difficulty of the problem and how big of a success it is to achieve even that level of confidence, but that low level of confidence really shows how far we are from being able to simply trust computer vision. Still useful from an information retrieval perspective, I expect.
Re:Cloud Video Intelligence API by muzakthings on Wednesday March 08, 2017 @04:24PM (#13823944)
You realize that softmax scores aren't probabilities, right?

It's just a relative measure of confidence, scaled such that they all sum to 1.0.

Re:Cloud Video Intelligence API by imh on Wednesday March 08, 2017 @08:57PM (#13826153)
That's kinda true, but (regularization aside) for standard loss functions it's minimized at the point it's well calibrated, right? Given the scores in the image (97% animal, 90% tiger, etc) they seem to be binary classifiers e.g. "is this a tiger?" So of all scores in the neighborhood of 90%, 90% should be "yes it is," making it a measure of confidence compatible with probability.

Please someone correct me if I'm wrong, but I'm pretty sure that's how it works, just like how logistic regression gives you a probability.

Re:Cloud Video Intelligence API by chipperyman573 on Wednesday March 08, 2017 @04:46PM (#13824145)
You can't add any of the numbers in the picture to equal 1.0 (or 100)
Re:Cloud Video Intelligence API by visarga on Thursday March 09, 2017 @01:27AM (#13827457)
What you need to do is to take the top prediction and see how accurate it is compared to a test set. The scores on the picture represent confidence not accuracy.