Tuesday, February 15, 2011

Humanity in Jeopardy!?

I've been watching the game show Jeopardy! since the early days of Art Fleming hosting it, something like 40 years! I would immodestly claim to be pretty good at its particular brand of diverse trivia, word association, and punning punditry (though my recall is not as instantaneous as it was once), and I've always wanted the chance to get on there.

So, the special program this week, in which two of the greatest champions ever are taking on an IBM-programmed computer called "Watson", has a great deal of interest for me. The contest has completed two of three half-hour segments, and Watson is mopping the floor with the two humans.

I watched the PBS Nova program in which the development of Watson's Jeopardy prowess was investigated in more depth, and the programming is remarkable for its subtlety, its breadth, and its rapidity. It is that last factor which seems to account for the computer's overwhelming performance.

Its accuracy, particularly in coming up with fact-based answers (OK, questions), has been impressive, though not perfect. The problem the humans are having is getting beat to the buzzer. I have a bit of complaint here: the mechanics of who buzzes first has always been mostly hidden from the viewer. Supposedly, the buzzers are suppressed until our intrepid host Alex Trebek has finished reading the question, to avoid interrupting the flow of the show; I suspect that Watson has not been inhibited in that way (as it would be too hard to implement). There's also the mechanics of how the computer is fed the "answer", which is not going to be the same way that a human contestant would do it (visually, while hearing it).

In terms of accuracy, I wouldn't rate the computer higher than the humans--at least these two humans, who rarely get one wrong. In the final questions of the first day's Single Jeopardy!, and in today's first-round Final Jeopardy!, Watson made glaring errors (I notice that one flaw remains uncorrected--after Ken Jennings missed one clue, the computer gave the same erroneous response). It almost seemed as if Watson was tanking on a few to keep the match close and the ratings from doing the same, but that would be anthropmorphizing--giving the computer motivations which it would be unlikely their programmers could include.

Very interesting has been the graphic provided by Watson showing the top three possible answers considered by the computer and their evaluated probabilities of being correct. It's a well-designed entertainment, but not a fair contest, and the human contestants should not feel compelled to burst their hearts (or brains), John Henry fashion, in what will surely be a failed attempt to beat the machine.

1 comment:

Anonymous said...

I really like it when folks get together and share ideas.
Great site, keep it up!
My web page > jeopardy template