This seems to come around on a regular basis, so there's got to be something to it. To summarize my thoughts on it: 1) yes, there are tons of things that could be improved in it, 2) even with such improvements, it's a fairly limited tool, and thus, 3) you can't place any more importance on this than you'd place on, say, a resume.

The most useful part about this tool, in fact, is not the specifics, but to see how the candidate rates himself, and compare your rating of his skills and experience to his own. If your rating and his come out radically different, one of you has a problem. :-)