News for hackers. Stuff that matters.
Re:IBM pitched Watson as a revolution in cancer care, but it... by code4tee on Wednesday September 06, 2017 @06:53AM (#15182635)
IBM has some OK offerings but they've really screwed themselves with all this hyped up marketing around Watson. They've become somewhat of a joke within the real data science community and business executives are also increasingly becoming disillusioned when projects based on IBMs mystery black boxes fail to deliver.

The MD Anderson situation was a debacle of monumental proportions. Such claims on cancer care are quite unethical in my view given that it gave a lot of very sick people false hopes and diverted a lot of hospital attention and funding away from stuff that actually works.

Re:IBM pitched Watson as a revolution in cancer care, but it... by carlmr on Wednesday September 06, 2017 @08:31AM (#15183075)
We have a lot of IBM tools at my company, and they're all terrible. I always heard these great things about Watson. Thank you for setting this straight. My world view is healed.
Re:IBM pitched Watson as a revolution in cancer care, but it... by moocowtruck on Wednesday September 06, 2017 @09:10AM (#15183335)
Lots of IBM where I work too, can't say a single good thing either.. honestly Watson being a bust is not very surprising! The people who pay the bills get taken to fancy parties and dinners by IBM, they give us free "trainings"(sales pitches) where they try to convince us to use something that management has already bought into... Honestly it's all very terrible :(

My sentiment of IBM these days is they sell software which is purposely made bad...and people buy it because, well no one ever got fired for going with IBM.

Re:IBM pitched Watson as a revolution in cancer care, but it... by fredley on Wednesday September 06, 2017 @09:48AM (#15183606)
The pattern I've seen: IBM sells things that underdeliver, the solution is always to buy more IBM to fix the inadequacies of whatever it is you've bought already. The go-to line is "Oh that's a know problem with X 3.4, if you upgrade to X 4.5 with Y 2.7 that will fix that problem". This process never stops.
Re:IBM pitched Watson as a revolution in cancer care, but it... by kiddico on Wednesday September 06, 2017 @11:47AM (#15184561)
Which is partly due to their actual problem solving strategy. Instead of fix the problem with X 3.4, they bought the company that makes Y so they could use that solution. But Y has it's own problems of course. And even if it didn't X should have just been developed along with something like Y. Though it's unlikely that X was even made by IBM anyways.
Re:IBM pitched Watson as a revolution in cancer care, but it... by virmundi on Wednesday September 06, 2017 @09:15AM (#15183368)
IBM workflow was surprisingly well thought out. You could model fluid flows through humans and services. Had a good deployment model I that the flows were scripts unlike IBM WPS 7 which had them embedded in EARs.
Re:IBM pitched Watson as a revolution in cancer care, but it... by JPKab on Wednesday September 06, 2017 @10:45AM (#15184070)
What's been crazy to me has been discussions with dudes I know who work there telling me that Watson branding has encroached on Cognos offerings and even disrupted sales.
Re:IBM pitched Watson as a revolution in cancer care, but it... by mipmap04 on Wednesday September 06, 2017 @10:49AM (#15184104)
They've been putting Watson on everything. Even Silverpop now looks like a Watson product.
Re:IBM pitched Watson as a revolution in cancer care, but it... by yread on Wednesday September 06, 2017 @07:18AM (#15182732)
> On a recent morning, the results for a 73-year-old lung cancer patient were underwhelming: Watson recommended a chemotherapy regimen the oncologists had already flagged.

> “It’s fine,” Dr. Sujal Shah, a medical oncologist, said of Watson’s treatment suggestion while discussing the case with colleagues.

> He said later that the background information Watson provided, including medical journal articles, was helpful, giving him more confidence that using a specific chemotherapy was a sound idea. But the system did not directly help him make that decision, nor did it tell him anything he didn’t already know.

I don't know, that doesn't sound bad at all. Perhaps it's just a case of overhyping. But if a computer can do recommend the same treatment as an expensive oncologist with years of training that's actually pretty good!

On the other hand it is quite baffling why doesn't FDA require a clinical trial or any independent review of its capabilities before allowing its use in clinic. It's a bit like Boeing certifying its own planes, I guess.

Re:IBM pitched Watson as a revolution in cancer care, but it... by jessriedel on Wednesday September 06, 2017 @08:44AM (#15183138)
> why doesn't FDA require a clinical trial or any independent review of its capabilities before allowing its use in clinic.

Because it's a tool for informing doctors, not a medical treatment. We don't have a clinical trial before PubMed rolls out a new search form either.

Re:IBM pitched Watson as a revolution in cancer care, but it... by dekhn on Wednesday September 06, 2017 @09:14AM (#15183362)
It's definitely part of medical treatment. Laws are changing to include "software medical devices" in health regulation. The FDA recently recognized this as a gap.

BTW I've worked with regulated software and it's not pretty. Just to update the firmware you have to fill out a ton of paperwork (because somebody died once after a firmware update and the firmware update was found to be the root cause). Regulation and review doesn't really make things tremendously safer, it just provides an audit chain so you can point at something when it does go pear-shape

Re:IBM pitched Watson as a revolution in cancer care, but it... by ska on Wednesday September 06, 2017 @09:39AM (#15183554)

   Laws are changing to include "software medical devices" in health regulation. The FDA recently recognized this as a gap.
This isn't quite correct - FDA has always regulated software medical devices. It is true however that the FDA is looking a lot closer at software than they did in previous decades, and is releasing guidance (e.g. recently on cybersecurity) that signals intentions moving forward. They definitely have signaled that they don't believe they were focused enough on software.

It's important understand that FDA approval to market is always done in terms of indications for use. You don't approve a device full stop, you approve it (and evaluate it) in terms of the indications. Only in this context can you determine if you have a 510k path (i.e. short cut) or need a PMA (i.e. clinical trial). It is also important to realize that different panels in the FDA (i.e. the type of device you are making) have different evaluation criteria.

So "Watson" is unlikely to ever be cleared per se (it is a tech platform) but you are likely to see individual Watson enabled systems file 510ks for particular indications. If I recall correctly this has already happened.

Re:IBM pitched Watson as a revolution in cancer care, but it... by ltowel on Wednesday September 06, 2017 @01:18PM (#15185365)
Do you remember what it's already happened for? Do you know what devices they're equivalent to?

While I get the impression FDA and industry prefer 510ks as much as possible, personally I don't see how this is close enough to anything out there right now for that to pass the smell test.

My guess would be that some `Watson` equivalent stuff will be 510k'd in as part of an MRI, CT or DBT imaging system or a CAD tool working on these systems. I think it'd take a while though, I don't think the regulatory science is there yet for actually showing value.

Re:IBM pitched Watson as a revolution in cancer care, but it... by ska on Wednesday September 06, 2017 @03:26PM (#15186412)
Off the top of my head, I don't, but I'll have a look if I have a chance later.

There have been approved CAD systems separate from imaging systems since the 90s.

"Value" is a nebulous term. FDA looks specifically for "safety & efficacy", and the criterion for each are rather dependent on the intended use.

Re:IBM pitched Watson as a revolution in cancer care, but it... by jessriedel on Wednesday September 06, 2017 @09:22AM (#15183436)
I can't tell if you're arguing that it is currently regulated, that it will be, or that it fits some platonic notion of 'medical treatment' and therefore should be (even if it's not beneficial).
Re:IBM pitched Watson as a revolution in cancer care, but it... by mindcrime on Wednesday September 06, 2017 @12:44PM (#15185057)
I've seen my doctor consulting Wikipedia before, along with other online resources. I don't think the FDA regulates that type of thing because it's just information, which the doctor is free to use or discard. I would think Watson or any other kind of medical diagnosis system would fall into the same category, as long as the human doctor is making the final decisions.
Re:IBM pitched Watson as a revolution in cancer care, but it... by pixelmonkey on Wednesday September 06, 2017 @08:05AM (#15182948)
> Hospitals pay a per-patient fee for Watson for Oncology and other products enabled by the supercomputer. The amount depends on the number of products a hospital buys, and ranges between $200 and $1,000 per patient.

Sigh. Let's remember that in medicine (a severely resource-constrained system), costs operate in a zero-sum environment. $200-$1,000 spent on Watson is $200-$1,000 not spent on some other treatment or human labor. And, in our current environment, specialist doctors will always have to re-do the diagnosis suggested by Watson, because guess who is held legally responsible, in the case of a malpractice suit? Not IBM, but the doctor co-signing the "recommendation".

Re:IBM pitched Watson as a revolution in cancer care, but it... by danmaz74 on Wednesday September 06, 2017 @07:17AM (#15182731)
TLDR: Watson has been trained for 6 years to build a knowledge base about 7 types of cancer; therapy recommendations are entered by experts from one top US hospital.

Looks to me like a giant pattern recognition system. The best assessment I found in the article:

> He said later that the background information Watson provided, including medical journal articles, was helpful, giving him more confidence that using a specific chemotherapy was a sound idea. But the system did not directly help him make that decision, nor did it tell him anything he didn’t already know.

Re:IBM pitched Watson as a revolution in cancer care, but it... by carlmr on Wednesday September 06, 2017 @08:41AM (#15183125)
>Looks to me like a giant pattern recognition system.

Pretty much all attempts at AI are pattern recognition systems. I think the AI hype will die down a bit, after we see more of the high profile AI programs underdelivering on marketing's promises. But we'll get some gems that do deliver something useful.

I just guess IBM bit off more than they can chew on this one. It's an old, slow, uninnovative company, trying to do what the cool kids are doing by putting all of their money on it.

Re:IBM pitched Watson as a revolution in cancer care, but it... by romaniv on Wednesday September 06, 2017 @10:41AM (#15184041)
>Pretty much all attempts at AI are pattern recognition systems.

All attempts in modern AI. Most interesting research from the 60s and 80s wasn't based on pattern recognition.

Re:IBM pitched Watson as a revolution in cancer care, but it... by danmaz74 on Wednesday September 06, 2017 @11:09AM (#15184265)
> Pretty much all attempts at AI are pattern recognition systems

Pattern recognition is a fundamental component of intelligence, but not all attempts at AI are based on supervised learning (ie "simple" pattern recognition).

Re:IBM pitched Watson as a revolution in cancer care, but it... by fractallyte on Wednesday September 06, 2017 @12:58PM (#15185174)
More like an expert system - the kind that was around in the 1980s: https://en.wikipedia.org/wiki/Expert_system
Re:IBM pitched Watson as a revolution in cancer care, but it... by bane on Wednesday September 06, 2017 @10:01PM (#15188627)
Yay, and now we're back at expert systems again. If we had known the AI winter would have been the dark ages and we'd have to reboot civilization again maybe we would have kept some fires on for warmth.
Re:IBM pitched Watson as a revolution in cancer care, but it... by fredley on Wednesday September 06, 2017 @09:26AM (#15183469)
Cynical take, having worked in organisations that have bought into IBM: IBM is not into curing cancer, it's into selling, and in particular up-selling at enormous scales (government and big business). The primary goal of the Watson project has not been to cure cancer, or do anything in particular, but to get existing customers to buy into extremely expensive contracts that include 'Watson' in some way. All the hype, which it is now becoming apparent was artificially expanded to some extent, is just a means to this end. As with most (all?) of their other products, it'll be oversold, underdeliver, but lock organisations into even heftier contractual chains.
Re:IBM pitched Watson as a revolution in cancer care, but it... by S_A_P on Wednesday September 06, 2017 @10:36AM (#15184012)
Its kind of sad to see what has become of IBM. It is now just another consulting shop that wants to plant a slew of 3-400/hr consultants to run projects. The big consulting companies best trick was to convince those in the enterprise that hiring a boutique consulting shop is "risky" and that a big 4/5 consulting company is a "safe" option even though it is orders of magnitude more expensive and not necessarily better, and in some cases worse when it comes to specialized systems.

Source: years of working in Energy/Commodity trading systems market and seeing several botched big 4 implementations that vendor/boutique shops can do at 1/10th the price.

Re:IBM pitched Watson as a revolution in cancer care, but it... by zwerdlds on Wednesday September 06, 2017 @10:42AM (#15184049)
Having done big and small consulting work, I think part of the issue is that waterfall still has quite a large buy-in. Smaller shops tend to go more agile, but when you have a team of analysts chomping at the bit to document your processes and say "yes, we can do this" (regardless of if they actually can) that builds confidence.

And that confidence is what moves the project forward, until the customer sees dates slipping, at which point it's too late.

Re:IBM pitched Watson as a revolution in cancer care, but it... by madenine on Wednesday September 06, 2017 @07:22AM (#15182756)
I was under the impression "Watson" was just a brand-face for IBM consulting services.

You're not buying thousands consulting hours and some big data services... you're buying Watson

Re:IBM pitched Watson as a revolution in cancer care, but it... by mholmes680 on Wednesday September 06, 2017 @08:37AM (#15183108)
I'm on board with that if it means I don't have to pay for additional PVUs [0].

0: https://www-01.ibm.com/software/passportadvantage/pvu_licens...

Re:IBM pitched Watson as a revolution in cancer care, but it... by notfromhere on Wednesday September 06, 2017 @10:09AM (#15183772)
As per every thread Watson is brought up in, it's mainly a catch-all term for its consulting division. IBM's consultants aren't very good, and the good ones move on to brighter futures.

This is why Watson is a disappointment and every Watson project ends in cost overruns and failure.

Re:IBM pitched Watson as a revolution in cancer care, but it... by arikrak on Wednesday September 06, 2017 @08:14AM (#15182987)
It's interesting that Watson appears to rely so much on processing text, which runs into so many difficulties. It would be helpful if (for a given disease) every patient's medical records were tracked as structured data. Machine learning would then be able to find patterns of diagnosis much more easily.

(I guess companies like Flatiron Health are working in that direction.)

Re:IBM pitched Watson as a revolution in cancer care, but it... by ska on Wednesday September 06, 2017 @09:46AM (#15183595)
People have been working on this for 30 years, and the mess keeps getting worse. A significant amount of that mess has been created by engineers trying to solve the wrong problems, or the wrong parts of them.
Re:IBM pitched Watson as a revolution in cancer care, but it... by gipp on Wednesday September 06, 2017 @08:50AM (#15183186)
Re:IBM pitched Watson as a revolution in cancer care, but it... by carlmr on Wednesday September 06, 2017 @08:42AM (#15183127)
But then all the NoSQL people working on data have to learn SQL ;).
Re:IBM pitched Watson as a revolution in cancer care, but it... by equalunique on Wednesday September 06, 2017 @10:22PM (#15188702)
Funny that you mention NoSQL. MUMPS, the original 1960s database system designed for medical data was NoSQL: https://motherboard.vice.com/en_us/article/3dkmg3/meet-mumps...
Re:IBM pitched Watson as a revolution in cancer care, but it... by cs702 on Wednesday September 06, 2017 @09:43AM (#15183577)
When results from "AI" don't match the hype nor justify the cost and hassle... that's how AI Winters begin.
Re:IBM pitched Watson as a revolution in cancer care, but it... by program_whiz on Wednesday September 06, 2017 @10:11AM (#15183791)
There's a bigger issue at hand here. AI generally has not made huge inroads into disease treatment. I think this is because we haven't figured out a good way to describe the diseases. Where it does work, we have a good way to classify the data (i.e. a set of training images or clear metrics to feed it). As Google/Verily learned the hard way, its not easy to fit digital models to organic beings.

In the area of cancer treatment, my guess is that the reason specialists are highly paid/trained is because the analysis and judgement required is highly manual and varied. Surgical intervention, prescription medications, targeted radiation treatment machines, patient psychology and counseling. Other than maybe guessing the prescription based on symptom list, what could the AI profitably help with?

Re:IBM pitched Watson as a revolution in cancer care, but it... by tim333 on Wednesday September 06, 2017 @11:27AM (#15184418)
>not made huge inroads into disease treatment. I think this is because we haven't figured out a good way to describe the diseases.

Perhaps more likely, human doctors are actually quite good and AI has yet to out do them.

Re:IBM pitched Watson as a revolution in cancer care, but it... by rezlov on Wednesday September 06, 2017 @12:52PM (#15185114)
> On a recent morning, the results for a 73-year-old lung cancer patient were underwhelming: Watson recommended a chemotherapy regimen the oncologists had already flagged.

> He said later that the background information Watson provided, including medical journal articles, was helpful, giving him more confidence that using a specific chemotherapy was a sound idea. But the system did not directly help him make that decision, nor did it tell him anything he didn’t already know.

This type of thinking is reason true ML adoption is still sparse. Analytics should be viewed as a supplement to decision making, not a replacement.

Manipulative marketing tactics by IBM and others are largely to blame. I'm confident that over time, as the general population becomes better educated on the topic, the perspective will shift.

Re:IBM pitched Watson as a revolution in cancer care, but it... by hacker_9 on Wednesday September 06, 2017 @08:15AM (#15182990)
> “It’s been a struggle to update, I’ll be honest,” said Dr. Mark Kris, Memorial Sloan Kettering’s lead Watson trainer. He noted that treatment guidelines for every metastatic lung cancer patient worldwide recently changed in the course of one week after a research presentation at a cancer conference. “Changing the system of cognitive computing doesn’t turn around on a dime like that,” he said. “You have to put in the literature, you have to put in cases.”

Wow the way they describe Watson is just like a massive book of documentation, with some NLP functionality layered on top to interface with the doctors. Keeping the database up to date must be a huge undertaking. Also this:

> "Given the same clinical scenario, doctors can — and often do — disagree about the best course of action, whether to recommend surgery or chemotherapy, or another treatment. Those discrepancies are especially wide for second- and third-line treatments given after an initial therapy fails, where evidence of benefits is slimmer and consensus more elusive."

Having an expert doctors opinion hard coded into the machine sounds good, but it won't be as good as a doctor in the room who can pick up on all sorts of other cues that I'm not sure how'd you even type into a computer. Things like breathing patterns, reaction to tactile stimuli, fuzzy logic ('that hurts, that hurts more'), facial expressions and so on.

Seems like this project has a long way to go, to get to the standard the IBM marketing team are pretending it's already at.

Re:IBM pitched Watson as a revolution in cancer care, but it... by teknover on Wednesday September 06, 2017 @09:03AM (#15183282)
"He noted that treatment guidelines for every metastatic lung cancer patient worldwide recently changed in the course of one week after a research presentation at a cancer conference."

Isn't this a good thing then that there's a digital assistant to ensure doctors are up-to-date and provide the right treatment plans?

Re:IBM pitched Watson as a revolution in cancer care, but it... by rscho on Wednesday September 06, 2017 @03:37PM (#15186498)
A digital assistant that signals guideline changes does not require advanced technology. This reflects failure of the medical system, more than anything else. Plus, the fact that guidelines change suddenly after some conference should tell you that treatment in the case depicted here is defined entirely on expert opinion. AI should seek to do better than just following expert opinion. Conclusion: the Watson system is a failure.
Re:IBM pitched Watson as a revolution in cancer care, but it... by hacker_9 on Wednesday September 06, 2017 @09:47AM (#15183601)
Of course, but in the very next sentence the doctor says the system stays out of date for quite a while. That's probably because they need an engineer to find all the old information and remove it, then input all the new information in whatever standard Watson requires.
Re:IBM pitched Watson as a revolution in cancer care, but it... by dep_b on Wednesday September 06, 2017 @07:28AM (#15182775)
I guess one day IBM will sue Scott Adams for plagiarism.
Re:IBM pitched Watson as a revolution in cancer care, but it... by gexla on Wednesday September 06, 2017 @10:50AM (#15184107)
IBM is nowhere close, but is there anyone closer?
Re:IBM pitched Watson as a revolution in cancer care, but it... by gregman1 on Wednesday September 06, 2017 @07:05AM (#15182680)
Who uses Watson besides IBM?
Re:IBM pitched Watson as a revolution in cancer care, but it... by arkadiytehgraet on Wednesday September 06, 2017 @07:26AM (#15182767)
All those poor developers, whose management bought into the standard IBM marketing lies and false promises.
Re:IBM pitched Watson as a revolution in cancer care, but it... by Angostura on Wednesday September 06, 2017 @07:59AM (#15182927)
Sir Arthur Conan Doyle?
Re:IBM pitched Watson as a revolution in cancer care, but it... by pamqzl on Wednesday September 06, 2017 @06:37AM (#15182573)
IBM seems to have painted themselves into a bit of a corner by calling a bunch of largely unrelated research projects by the name Watson.
Re:IBM pitched Watson as a revolution in cancer care, but it... by ralfn on Wednesday September 06, 2017 @08:26AM (#15183050)
IBM is doing the smoke and mirros and selling bullshit game for a while now.

And it's intentional! They have a strategy of inventing new words that sound all scientific (like 'cognitive computing') but there is absolute no meaningfull research from them. Its like the whole company has become a parody.

Any companies that allows their employees to talk to IBM has the problem that afterwards their employees understand less of actual problem domain -- and the knowledge they think they gained from that interaction .. its all 'IBM and consultancy buzzwords' rather than the industry-standard terminology fueled by actual research and science.

Re:IBM pitched Watson as a revolution in cancer care, but it... by devdoomari on Wednesday September 06, 2017 @08:25AM (#15183046)
+1 if I were in IBM's position, I'd just be quiet about everything until it's quite 'safe' for publicity...

but on the other hand, IBM's negotiation with MD anderson was one kind of a deal - instead of paying for hospital data, IBM made hospital pay...

Re:IBM pitched Watson as a revolution in cancer care, but it... by georgeecollins on Wednesday September 06, 2017 @09:31AM (#15183509)
I don't like the Watson logo. It's too busy. If you shrink it or squint when you look at it you just see a blob.
Re:IBM pitched Watson as a revolution in cancer care, but it... by mholmes680 on Wednesday September 06, 2017 @10:33AM (#15183983)
The old one or the new one? I like the subtle "heart" shape in the new one, especially the animation, to implicitly persuade me to love IBM.

http://www.underconsideration.com/brandnew/archives/new_logo...