Watson crushes the competition in second round of 'Jeopardy'

Re:Watson crushes the competition in second round of 'Jeopardy' by ellyagg on Wednesday February 16, 2011 @12:47AM (#2225214)

It is certainly unfair and has pretty much ruined the competition for me. Watson can instantly detect when he's allowed to ring in. This is no different than a car "racing" a human. In theory, a human contestant can try to time his trigger to coincide with the end of Trebeck speaking the question, and catch the light just as the timing person releases the question, but in practice this is fraught with peril--if you're off you can't ring in for several tenths of a second--and requires much more of a human's energy, which he can no longer devote to figuring out the question. You could see Jennings attempt this on the first day, with mixed results. These episodes are commercials for IBM and it's clear who is supposed to win.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by gloob on Wednesday February 16, 2011 @12:56AM (#2225250)

Not certain what's unfair about it - if we accept that the human contestants would have scored comparably to Watson (if they had the opportunity), then the competition really does boil down to who has the fastest thumb. I can't imagine it would be any more entertaining to watch IBM throw the match by intentionally crippling their player.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by zach on Wednesday February 16, 2011 @02:41AM (#2225553)

Instead, the game has been crippled. Watson has to press the button, naturally, but they're not going to require Watson to write out the answer with a pen, of course. And forget about audio clues, much less a picture-based Final Jeopardy clue.

I mean, if they had changed the ring-in system to work in a manner befitting a man-versus-machine match, you're telling me that wouldn't be entertaining? I would be fascinated by that match. This one was a letdown.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by ericb on Wednesday February 16, 2011 @08:09AM (#2226200)

Watson would likely lose if this was instead of jeopardy, a simple trivia test where contestants were scored for the most correct answers. Thinking about that really calls the contest into question for me. We let the machine off the hook for things people are good at but it is not (understanding speech, reading and vision) but we make no allowance for a machine's ability to press a button faster than humanly possible. If you instead phrased the contest as "machine presses button faster than humans" it doesn't seem impressive.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by gloob on Wednesday February 16, 2011 @10:23AM (#2226674)

Your implicit assertion that natural language processing is something "a machine is good at" really calls the rest of your post into question for me. In particular, your demand that the researchers solve half-a-dozen Hard Problems instead of only one seems mildly mean-spirited.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by baddox on Wednesday February 16, 2011 @01:05AM (#2225282)

It's not unfair exactly, but it certainly ruins the impressiveness of Watson's ability to come up with the right answers. I highly doubt Watson's accuracy is even close to Brad or Ken's, but the buzzer seals the deal.

It's less impressive, just like a computer sorting 1000 integers faster than a human is less impressive.

edit: added "not" after the first word

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by gloob on Wednesday February 16, 2011 @01:09AM (#2225292)

From the descriptions I've read, Watson only buzzed in after it believed it had a solid answer. If it was significantly less accurate than its competitors, it wouldn't have spent the entire match buzzing in before them.

I'm afraid I don't really understand the decision to trivialize the fact that we now have a computer that can answer general-knowledge natural-language queries quickly and about as accurately as a clever person. That's a Big Deal.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by baddox on Wednesday February 16, 2011 @01:27AM (#2225354)

For every single clue, there is an overlay showing Watson's top 3 answers, and it shows whether Watson was confident enough to buzz in. In day 2, I counted only 3 times when Watson had a confident response but was beaten to the buzzer.

None of this really minimizes IBM's accomplishment, but it absolutely means this specific presentation (Jeopardy!) lacks weight for those of us who understand what the game dynamics of Jeopardy! are. This is nowhere near as impressive a "man vs. machine" victory as was Deep Blue vs. Kasparov.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by ugh on Wednesday February 16, 2011 @03:31AM (#2225670)

Even considering that, I don’t know what exactly makes it less impressive than Deep Blue. Watson correctly knew 84% of the answers in the second round. Looking at past games, that’s very much competitive with humans. To me, this is much more impressive than Deep Blue, even considering the fast reaction times.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by jarin on Wednesday February 16, 2011 @03:29AM (#2225665)

Here's the thing, a tiny microchip sorting numbers at a speed that a human can barely comprehend IS impressive.

Just like flying through the sky in a metal tube, talking to someone on a different continent in realtime, or converting lightning into high-energy photons and cooking your food with it.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by glhaynes on Wednesday February 16, 2011 @09:26AM (#2226425)

Well said. I must say, I don't really relate to the "man vs. machine" narrative here. Maybe it's just because I'm not taking the long view (the one in which the machines inevitably rise up and enslave us? or in which "we" become machines?), but all I see is wonderful positives: as you say, the fact that we can now fly in metal tubes doesn't diminish us as humans for not having wings and jet engines. Look at us humans: we can fly now!

I really enjoyed this article from Garry Kasparov in the New York Review of Books. Spoiler: it's [partially] about how the best chess player in the world is a really good human paired with a really good computer. http://www.nybooks.com/articles/archives/2010/feb/11/the-che...

So let's all join hands with the machines and sing Kumbayah ... all watched over by machines of loving grace... of course, yes, this is a medium-term view. The long-term, I suppose, probably belongs to the machines.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by baddox on Wednesday February 16, 2011 @01:36AM (#2225375)

You're completely right, and I'm sick of people equating minimizing Watson's Jeopardy! success with minimizing the software/hardware accomplishment behind Watson. IBM has done something great here, but the buzzer dynamic renders Watson's Jeopardy! prowess unimpressive.

A good analogy, similar to your car vs. human analogy, is this: Hold a competition where two contestants much first pass some sort of Turing-like test. Upon passing that test, both contestants must sort one million 32-bit integers. The first contestant to finish both tasks wins.

In that hypothetical competition, it would certainly be impressive for a computer contestant to pass the Turing-like test. (The human contestant would probably have no trouble doing so.) But the sorting task would seal the door, and the computer would win every time it is able to pass the Turing-like test. The sorting task is known to be dominated by computers, just like buzzer reflexes are known to be dominated by computers.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by ugh on Wednesday February 16, 2011 @03:32AM (#2225675)

I don’t know what exactly makes Watson’s performance unimpressive. Being able to answer 84% of all questions correctly is impressive, reaction times or not.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by baddox on Wednesday February 16, 2011 @03:37AM (#2225686)

As I've said many times, Watson being able to come up with correct responses that often is impressive. Watson winning Jeopardy! is what's not impressive.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by ugh on Wednesday February 16, 2011 @03:39AM (#2225691)

That seems to be a contradiction. Being able to answer 84% of the questions seems to be about champion level.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by baddox on Wednesday February 16, 2011 @10:43AM (#2226774)

It might be close to champion level, but I strongly suspect that it's not as good as Watson's two human contestants.

[ Parent | on HN ]

Re:Watson crushes the competition in second round of 'Jeopardy' by ugh on Wednesday February 16, 2011 @10:46AM (#2226793)

I would be interesting to know how the human contestants would play on their own. We only see them in competitive matches, so we don’t really know how many of the questions they would and could answer. (I heard that contestants tend to push their buzzer as soon as they hear one of their competitors pushing the buzzer regardless of whether they know the answer, so you can’t even know from buzzer presses alone whether the would answer if there were no competitors.)

84% is, to my mind, very high. I’m skeptical of claims that even the best humans are better than that.

[ Parent | on HN ]