Re:Watson crushes the competition in second round of 'Jeopardy'
by baddox on Wednesday February 16, 2011 @10:51AM
(#2226826)
With humans, you don't know how many correct responses they knew. You only know how many triple stumpers there were and how many incorrect responses a human gave. When two or three human champions are playing, it's unlikely for one of them to buzz in first with the consistency that Watson was able to. Hypothetically, if two humans A and B both knew the same 95% of correct responses, you would probably see something like A buzzing in 30% of the time and B 70%. You couldn't possibly determine how many correct responses either A or B knew.
[ Parent
| on HN ]
Re:Watson crushes the competition in second round of 'Jeopardy'
by ugh on Wednesday February 16, 2011 @10:57AM
(#2226866)
I didn’t try to find that out, it’s, as you say, impossible to find out. Triple stumpers set an upper boundary for the humans, the theoretical maximum. That’s what I calculated. About three triple stumpers per round seem to be the norm, even among the best, that puts the upper boundary – the theoretical maximum – at 90%. The humans might still be worse than 90% but they are definitely not better (when there are three triple stumpers).