After all, winning the lottery is a lot less impressive if you've bought a few million tickets and this article says absolutely nothing about any kind of controls or whether or not this was a one-off performance or if it would work at scale.
FWIW, here's Lukas' writeup of his experience.
They're definitely not free; doctors have to evaluate what Watson says, and if Watson is wrong a significant fraction of the time, then doctors will spend a lot of time and energy barking up the wrong tree.
I thought the 20 million oncology studies Watson cross referenced was the control and I arrived at the opposite conclusion as you about the usefulness at scale.
For example, I immediately thought that under the Affordable Care Act, the number of patient Electronic Health Records the Government collects eclipses what Watson had to work with in this case so at scale the potential is huge. Admittedly, until reading your comment I did not consider there were likely a number of times Watson misdiagnosed (which seems obvious now). Therefore, it would be interesting to rerun against the Electronic Health Records the Government has collected so far and test if Watson could diagnosis correctly in less attempts than with the 20 million record it used (or what I call the control). Though at that point, I think it would be the patient that is the control.
I don't know, I'd still be impressed, wouldn't you? Buying a few million lottery tickets is impressive by itself, both logistically and financially. And even still, odds. It's a lot of risk for anybody to dangle.
Shit, buying every ticket is even more impressive. (I grant your point, you just posed a funny metaphor as comparison.)
Tests like these are all about the statistics and without statistics this result - while important to this particular individual - is almost meaningless.
That's cynical.
Edit: And when the hell did cynical become a bad word in silicon valley? I've had a number of posters on here respond to things "oh, stop being cynical!" as if that was some kind of contribution...
I've got a word for people who are cynical.
Decent scientists.
Feynman would be rolling over in his grave...
"A skeptic doesn't believe anything without strong reasons, which is why it is also associated with doubt, especially when something hasn't been experienced yet. Cynicism is believing the worst of something or someone. It has nothing to do with evidence. It is an outlook on life."
If you understand Bayesian reasoning, then we all have unjustifiable prior beliefs, perhaps due to innate personality, or due to our own experiences.
Calling someone a cynic is often just a way to bash someone who has different priors.
The better thing would be to ask where those priors come from, no?
> Its not cynical if you've actually had any dealings with IBM and Watson.
Maybe GP knows something you don't and he's talking from experience?
The applicability of either moniker towards anyone in this debate, or any other, is up to the participants to hash out.
No, that's history. IBM has quite a series of Watson 'breakthroughs' that ultimately went nowhere, to the point where when I see 'Watson' mentioned in an IBM press release it automatically gets discounted if there isn't a part where the subject has some statistics to go with the inevitable hype.
I see it as IBM trying to stay relevant in a world that needs it less and less and that's why they focus on anything that will grab headlines.
Winning jeopardy, curing cancer, what's not to like about IBM?
http://webcache.googleusercontent.com/search?q=cache:GaC_y_3...
Cached link supplied because the original has departed.
One possibility is that Seishi Ogawa's group has figured out that something off-label can more effectively treat cohesin mutants and it is under review. But I'm not sure how Watson would have known about that. Interesting.
Probably my reading is flawed, but this doesn't make sense.
John Welch, Rick Wilson, Tim Ley and colleagues showed how to do this several years ago: http://jama.jamanetwork.com/article.aspx?articleid=897152
I'm curious whether this case was confounded by cytogenetic complexity, which was part of the problem in that case. Ley later diagnosed Lukas Wartman, a fellow in his lab (or, more like, they worked together to figure out what to do), too:
http://oncology.wustl.edu/people/faculty/Wartman/Wartman_Bio...
Lukas did not look nearly as healthy as he does in that picture when I saw him last. He is a great scientist and what he had (adult acute lymphoblastic leukemia, ALL) is a nasty malignancy to treat. Children with ALL tend to do well, but their disease typically seems to arise from different underlying causes than adults, who do poorly.
(Not-so-ninja edit: Here is Lukas' own writeup of his experience. He was in a much tighter spot than I recalled: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850894/ Even children in second relapse are viewed as bad news. Adults in second relapse are generally considered dead men walking. Anyways, read the paper if you're curious.)
If Watson caught that the woman had (say) Ph+-like ALL and suggested the right TKI, that will impress me quite a bit, because I've seen someone die from a wrong guess on the latter. On the other hand, if it was APL, they fucked up the differential somehow and are probably kicking themselves.
(Double not-so-ninja edit: It wasn't APL-vs-AML, or ALL subtyping. Another poster kindly pointed out the primary source for this news, and something doesn't add up.shrug)