News for hackers. Stuff that matters.
Re:The Stupidity of Computers by adwf on Friday February 07, 2014 @11:17AM (#7196961)
Yes 87% is quite impressive, but that remaining 13% is all important. The errors sadly propagate (as above); more importantly, the errors tend to strike randomly also.

eg. A human might not be expected to know too much medical terminology, hence could easily get names/locations/functions of muscles and organs mixed up. Whereas a computer might be brilliant at that, but then suddenly not know what a door knob is (door handle? door knob of butter? etc...)

So ultimately to make a computer seem intelligent the problem is two-fold: Getting a high percentage of right answers, whilst also getting the "right" wrong answers.

Afterall, we don't criticize a human too much for not knowing something like the exact geographic location of a small African nation. But we definitely would if they thought that it was on Mars! It's exactly the same with computers, they're just far more likely to make wildly incorrect mistakes than understandably incorrect ones. So finding a way to mitigate computer mistakes, to make them milder, is every bit as important as eliminating them entirely - a potentially impossible task.

Re:The Stupidity of Computers by blueblob on Friday February 07, 2014 @02:23PM (#7198164)
If you think about the way that humans learn, we get formal education that the computer doesn't get. We are getting corrected for our mistakes by other people. This work did not originally do that. The newer versions of it get corrected by people but that doesn't mean that it couldn't be corrected by more instances of itself that operate over different datasets via a consensus algorithm. The algorithm will also probably get flack that it is not unsupervised when human learning is actually partially supervised.

One could argue that humans have a similar problem of errors propagating. Areas that we feel strongly about can bias us against learning in fields such as religion, politics, and, of course, programming language design.

I think that it is possible that humans would make some similarly poor responses in areas that we don't talk about often.

Re:The Stupidity of Computers by adwf on Friday February 07, 2014 @05:03PM (#7199067)
I agree absolutely. I believe that future AI's will be educated in much the same way we educate children. Perhaps less education will be needed - possibly we could start them off at a higher age bracket for example. But ultimately I'm certain the first "strong" AI's will be educated/supervised to some degree.

My point was more that perhaps we're a bit too focused on the wrong metric for success. The current criteria set is {positives, negatives, false positives, false negatives}, and we try to optimise for high/low degrees of one or the other in order to determine whether a particular approach is successful or not.

What is then overlooked, is that perhaps we don't need to have a near-perfect positive rate, but instead achieve an acceptably-incorrect false positive or false negative rate. Where the answer may be wrong, but it's not too far wrong. Much like a human might pin a country like India in the wrong place on the map, but wouldn't ever put it in the middle of the Indian ocean.

In summation: Perhaps the key for computers to appear intelligent, is not to be perfectly correct, but to be not too disastrously incorrect.

Re:The Stupidity of Computers by blueblob on Friday February 07, 2014 @08:25PM (#7199895)
Perhaps you can extend the F1-score[1] to be an F1,w-score that weights errors based on some measure of distance from correctness.

[1]http://en.wikipedia.org/wiki/F1_score

Re:The Stupidity of Computers by adwf on Friday February 07, 2014 @08:57PM (#7199979)
Sounds like a good idea. Finding that distance heuristic will be a challenge though!
Re:The Stupidity of Computers by cortesoft on Friday February 07, 2014 @03:22PM (#7198489)
I agree. In fact, errors like thinking an internet cookie is a baked good sounds EXACTLY like the type or mistake a small child would make.
Re:The Stupidity of Computers by 3rd3 on Friday February 07, 2014 @03:38PM (#7198575)
That makes me remember that as a small kid there were certain spots in my environment beyond which, I was conviced, infinite wilderness would adjoin or the world would just end (for example behind forests or hedges). Growing older I was often sobered when I discrovered there was just bog-standard urban landscape.

Yet, the machine seems to come from an entirely differernt direction. It has much more facts accumulated than a child and can articulate and process them to absolute precision.

Re:The Stupidity of Computers by Gracana on Friday February 07, 2014 @04:02PM (#7198707)
I remember visiting my childhood elementary school and seeing that the "forest" behind the building was just a scraggly patch of trees with a chain-link fence on the other side. It seemed so much bigger back then...
Re:The Stupidity of Computers by evunveot on Friday February 07, 2014 @01:58PM (#7198017)
So finding a way to mitigate computer mistakes, to make them milder, is every bit as important as eliminating them entirely - a potentially impossible task.

Seems like they should run multiple instances against different but overlapping corpuses and see what they agree on. For all we know, any single person would forget what a door knob is if they had all the information in the world in their brain.

But maybe it already works that way internally, I don't know.

Re:The Stupidity of Computers by waveman2 on Saturday February 08, 2014 @05:19AM (#7200848)
> they're just far more likely to make wildly incorrect mistakes than understandably incorrect ones

Have you spoken to some members of the general public and asked what they think? People believe all sorts of wacky things. We are just a lot more tolerant of the mistakes people make. http://www.buzzfeed.com/mjs538/things-americans-believe-in

Re:The Stupidity of Computers by varelse on Friday February 07, 2014 @01:33PM (#7197857)
13% error rate in a self-driving car, hilarity ensues...
Re:The Stupidity of Computers by adwf on Friday February 07, 2014 @05:10PM (#7199103)
Exactly. A human would usually make "acceptably incorrect" decisions, such as not indicating, or speeding. But these mistakes generally don't end in disaster. Only occasionally making a really bad mistake ending in an accident. So out of those 13%, it might only end up with <0.1% overall as accidents, the rest being recoverable.

Whereas the uncontrolled randomness of computer mistakes could be horrendous. Hence why I'm saying an interesting avenue to look at in AI, might be to achieve acceptably incorrect answers (acceptable negatives), rather than just aiming for high true-positive and low false-positive rates.

Re:The Stupidity of Computers by Raphael on Friday February 07, 2014 @04:27PM (#7198851)
Leave room for a margin of error, same as humans do.
Re:The Stupidity of Computers by Houshalter on Friday February 07, 2014 @09:54PM (#7200107)
Well reading through their site it seems it gets some categories of facts right nearly all the time, and some categories it does poorly at. This should be expected because some things are hard to understand even for a human, especially if you don't already know a lot about the subject. However it does seem to assign a confidence estimate to every fact it learns. So it does get some facts wrong, but at least it knows what probability there is the fact is wrong and doesn't assume it is 100% true.