News for hackers. Stuff that matters.
Re:Day 1: IBM Watson Ties for first on Jeopardy by ugh on Tuesday February 15, 2011 @02:24PM (#2222776)
Watson needs a few seconds to calculate the answer and will only hit the buzzer when it’s finished calculating. Its opponents can buzz in even if they are not completely sure that they have an answer (or they simply might be faster than Watson). It’s important to note that it is only possible to buzz in after the question has been read out. (Someone behind the stage flips a switch or something like that. Lights indicate that the buzzers are open for the humans, they are locked out for a few hundred milliseconds when they press the buzzer too early, Watson never buzzes in prematurely.)

I would like to know whether it is possible to beat Watson to the buzzer even if you and Watson both already know the answer. Watson has probably better reaction times but humans can anticipate when the host is finished reading. That probably depends highly on the consistency of the person behind the stage flipping the switch and opening the buzzers.

Here are some stats from the first round: Watson was the first to buzz in 16 times (with two wrong answers). It was above its buzzing threshold and didn’t manage to buzz in seven times. (It would have been wrong three times if it had made it to the buzzer first.) It was below its buzzing threshold six times. (The remaining one is the daily double which Watson got correct.)

This needs to be in a table. Numbers in brackets indicate wrong or potentially wrong answers:

  Confident & first:  14+(2)
  Confident & beaten:  4+(3)
  Not confident:       6
  Daily Double:        1
Edit: Some think that Watson’s (probably) superior reaction time gives it an unfair edge. I don’t really agree because fast reactions are simply a part of the game but I can sort of see the point that it’s not really about reaction times. We already know that computers can be better than humans when it comes to those.

I propose the following modification: One of the contestants who manages to buzz in in the first 100ms (the human reaction time) is randomly selected, buzzing in after those 100ms works as usual. Contestants are also no longer locked out for buzzing in too early.

Re:Day 1: IBM Watson Ties for first on Jeopardy by KVFinn on Tuesday February 15, 2011 @05:18PM (#2223621)
I'm a bit lazy to look up for the source, but one article mentioned that the engineers were surprised that humans could get down to the 10 - 15 ms range by anticipating the light rather than reacting to it. They said this beats the mechanical buzzer, and it does happen, though not terribly often.

So yes, it is possible to beat Watson to the buzzer even if you both know the answer.

I suspect they could have improved the buzzer if they really wanted to, but it might have made things too lopsided.