News for hackers. Stuff that matters.
Re:DeepMind readies first commercial product by melling on Sunday March 31, 2019 @11:12PM (#19540117)
AI is being developed to make MRI scans 10x faster.

https://www.forbes.com/sites/samshead/2018/08/20/facebook-ai...

If DeepMind is trained on millions of MRI’s, we might have better preventative medicine.

Re:DeepMind readies first commercial product by dontreact on Monday April 01, 2019 @12:40AM (#19540550)
I have talked at length with several people in the FastMRI project and in my opinion it is actually very dangerous. There is no feasible way to validate that such a model will not hallucinate normal tissue in the presence of a rare abnormality. The argument often used is that advanced reconstruction techniques such as compressed sensing have not required validating against very rare abnormalities, however when deep neural networks get involved you have a nearly universal function approximator whose behavior is nowhere near as bounded.

The FDA will probably set the low bar of validating that the reconstruction algorithm fares well in the face of a few abnormalities and call it a day, instead of the proper (admittedly infeasible) validation of testing against all abnormalities that the reconstruction algorithm will encounter. Facebook will probably happily jump over this low bar, and patients will get hurt.

Re:DeepMind readies first commercial product by LeanderK on Monday April 01, 2019 @06:46AM (#19542030)
I never understood these arguments. Isn't it like claiming self-driving cars via neural networks would never be possible because you can't test that the neural network would take the correct decision in every situation?

I view the whole issue stochastically, with the immediate aim being to make (significantly) fewer errors than the current approach, which is having a human decide. I don't claim that I can design an experiment which could selve as an indication whether we are improving upon human judgments, but I think this should be the goal.

Reflecting upon my view, i think it comes from the experience of training ml-algorithms. You are always minimizing errors, but you goal is almost never to make 0 errors, because often your data is noisy and you are probably overfitting. I know the medical enviroments are more sensitive, but I can't really wrap my head around how we could design a learning algorithms that does not make any error and works on all abnormalies. I think it will always missclassify.

Rephrasing my argument: I think the approval should be given if an significant expected improvement over the distribution of real-life abnormalies can be detected and not over the uniform-dsitribution over all abnormalies.

EDIT: detecting out-of-distribution samples is hard and I don't think this is a solution and leads to a false sense of security.

Re:DeepMind readies first commercial product by PavlikPaja on Monday April 01, 2019 @07:18AM (#19542199)
It isn't. It obviously is possible to drive a car. The problem here is you're trying to reconstruct image from less data than what is fundamentally possible. Compressed sensing is already as low as you can get, unless you prove it is possible to reconstruct the image accurately from even less data, then it's probably pointless to try use neural networks to decode it, and it's actually a bad idea anyway.

The problem with neural networks is that they can reconstruct something that looks "normal", not necessarily something that is accurate. The more abnormal the scan, the more likely it will get reconstructed as something that looks perfectly fine even when it's not.

Re:DeepMind readies first commercial product by LeanderK on Monday April 01, 2019 @07:38AM (#19542317)
I really don't know anything about MRI and their usage, i was mainly criticising that his arguments sounded like a lot of the arguments I've heard against other usages of ML-Algorithms.

What I meant: There's probably a medical reason why you want such a product and if a more readily available MRI saves (really significantly) more lives than the chance that it might miss some abnormalies which could lead to death, then I think we should allow it. That's what I meant with a stochastic view. If we, for example, only have a few scans per hospital available because the chance might exist that we missclassify something and lots of people get worse or delayed treatment because they are not high enough on the priority list to get access to a super-resulution MRI with a fidelity they don't really need (again, i don't know anything, just to illustrate my point), then I think something is wrong.

His argument just sounded dismissive without giving a, to my uninformed point of view, valid reason.

Re:DeepMind readies first commercial product by dontreact on Tuesday April 02, 2019 @10:31PM (#19560313)
the distribution of abnormalities should be weighted by the severity of a miss. Unfortunately there are many abnormalities with lethal outcomes if missed. So it’s very hard to get a sense if a reconstruction algorithm will adversely affect outcomes. I would appreciate even a more detailed look at trying to estimate this, especially in light of how much more a deep learning system can be. However that is not what’s happening. Everyone is happy to just jump over the bar that has been set for previous reconstruction algorithms.
Re:DeepMind readies first commercial product by tntn on Monday April 01, 2019 @01:36AM (#19540787)
I don't have strong feelings about this, but a few thoughts come to mind regarding exploring accelerated MRIs via neural networks.

1. If an MRI can be done 10x faster with the same results except in exceptionally rare cases, might that not still be a win? Order of magnitude reduction in time may translate to substantial reduction in cost and increase opportunity for applications. It seems like it is worth considering whether these benefits might be worth compromising the accuracy of the imaging.

2. How accurate are radiologists at diagnosing / detecting these rare abnormalities that validation might miss? If radiologists are actually pretty mediocre at this, might it be OK to make the scan slightly imperfect if the next stage (the human) is already very imperfect?

Re:DeepMind readies first commercial product by dontreact on Monday April 01, 2019 @02:20AM (#19540973)
1. Maybe, but that should be a tradeoff that is made consciously with some analysis and care, instead of just jumping over a low FDA bar, which is what everyone in this space seems to be doing. I think there are enough abnormalities that can be fatal which a net would just fill in (e.g. aortic dissection?)

2. There are thousands of different abnormalities. From what I understand about the FDA validation process for this sort of thing there would be only 10s of abnormalities. There are likely, many, many of them that are quite obvious to radiologists. And once again, this would be a question that should be studied carefully when people's lives are at risk, instead of just assuming that it will be fine then going ahead to "move fast and break things"

Re:DeepMind readies first commercial product by tntn on Monday April 01, 2019 @10:50AM (#19543811)
> And once again, this would be a question that should be studied carefully when people's lives are at risk

I agree 100% with this, but how can this question be studied of this research to develop these systems doesn't go forward? You earlier characterized the FastMRI research as, in your view, "actually very dangerous." As I see it, the research here is potentially extremely valuable, and the danger comes not from the research, but from somebody deciding to deploy it without considering these questions. Typically the road from research program to wide deployment is quite long, and I disagree that we should discourage the research because of potential flaws in the productization process. (Unless there is ill will on the part of the researchers, which I'm assuming there is not).

Re:DeepMind readies first commercial product by dontreact on Tuesday April 02, 2019 @10:32PM (#19560320)
I think it’s the lack of will to study these sort of things that is dangerous. There just doesn’t seem to be this sense of caution among the people working on this. They seem to just want to jump over the low bar set by the fda and move ahead to patient care.
Re:DeepMind readies first commercial product by bartimus on Monday April 01, 2019 @02:39AM (#19541041)
It's my understanding it's sometimes exactly those obvious abnormalities that are missed by radiologists.
Re:DeepMind readies first commercial product by nitrogen on Monday April 01, 2019 @05:48AM (#19541785)
Adding another lossy step will only make even more anomalies get missed, or worse, misdiagnosed as something else (I'm thinking about the photocopiers that sometimes change numbers in documents).

The argument that "our system is already bad so we should just merge this new bad component because it doesn't make it worse" is bad in software, and unacceptable in medicine.

Re:DeepMind readies first commercial product by tntn on Monday April 01, 2019 @10:41AM (#19543706)
I don't think that is a fair characterization of what I wrote.

If the scanning process has a sensitivity of 99.999%, and the next stage in the signal chain has a sensitivity of 50%, and we consider what happens if the MRI sensitivity drops to 99.9%, that's well in the noise in terms of diagnostic value. Use some of the extra money freed up by doing the MRI 10x faster to pay a radiologist to look at the scan for 10% longer, and perhaps the net accuracy has improved.

What is unacceptable about this kind of holistic reasoning about the system? High performing systems are frequently not composed of perfect components.

Re:DeepMind readies first commercial product by nitrogen on Monday April 01, 2019 @11:46AM (#19544563)
It's fine to talk about statistical accuracy when the distribution is somewhat normal. ML systems we've seen so far don't typically degrade in nice, statistical, predictable ways.

Omitting 90% of samples (based on 10x speed claim) leaves an awful lot of room for bizarre errors.

Re:DeepMind readies first commercial product by bartimus on Tuesday April 02, 2019 @07:00AM (#19552793)
One advantage of software is that it can't really have a bad day. The radiologist can be affected by external human factors. Perhaps an angry wife and a dinner getting cold at home. He might not take the extra picture a bit further down the lower back where the golfball tumor was growing (story from my mother-in-law).
Re:DeepMind readies first commercial product by skwb on Monday April 01, 2019 @12:51AM (#19540599)
To be honest I am highly skeptical that FB would be able to have the sort of institutional capacity to launch any sort of medical device health product. It takes a non trivial amount of effort and time to fully productize it (in terms of system integration/ product testing/ FDA submission). Unless there is involvement with one of the major MRI manufacturers, I don't foresee this going far. Many more players in the game that actually have a decent shot of getting something like this to market.
Re:DeepMind readies first commercial product by duado on Monday April 01, 2019 @01:49AM (#19540850)
The thing is that because the highest-ROI application of machine learning is online advertising, unquestionably the two deepest pools of machine learning talent are at Google and Facebook. So this may be able to overcome the obvious culture mismatch between "move fast and break things" and the FDA.
Re:DeepMind readies first commercial product by skwb on Monday April 01, 2019 @02:20AM (#19540972)
Google has a long long history of failing in the medical space. They tend to overengineer their products in a way that makes sense if you’re a computer engineer, but not if you understand anything about the healthcare system. Facebook has no legitimate history developing these products either. It’s more involved to develop health products than it is to sell ads. And when it comes to deep learning, optimizing your ml for 2D rbg is not the same beast as volumetric 3D data.

And besides, let’s say tomorrow they have a method to do it tomorrow: how do they prospectively scam patients? How do they deploy the algorithm in a clinical setting? For any of this as a product to work, it would have to be integrated into a mri controller. Unless I’m eating my words at RSNA this year and deep mind is presenting their work, I’ll remain highly skeptical that this is going anywhere beyond a PR story that’s been sold to the media.

Re:DeepMind readies first commercial product by throwawaw434234 on Monday April 01, 2019 @12:52AM (#19540603)
Deepmind has projects related to 'formal verification' and conservative bounding, presumably for these very reasons.
Re:DeepMind readies first commercial product by dzhiurgis on Monday April 01, 2019 @03:54AM (#19541359)
From what I understand there are tons of MRI applications. If you are looking for something exotic of course you will try to get a human, probably best in the field.

But if you are trying to bring this to masses one can start with with something "simple" like bones or some some relatively trivial organ. Not that MRI prices themselves are dropping much - gotta wait for room temperature superconductors.

Re:DeepMind readies first commercial product by dekhn on Monday April 01, 2019 @08:04AM (#19542465)
nobody expects these systems to be perfect, which is why they have less validation against extremely rarely observed problems- the total impact is small enough that we don't prevent good systems from being approved.

Rather than saying "very dangerous", it would be safe to say that some individuals woudl not be well-served, but ideally, the overall health of the population would increase for a reasonable expenditure.

Re:DeepMind readies first commercial product by krapht on Monday April 01, 2019 @12:28AM (#19540506)
"Using AI, it may be possible to capture less data and therefore scan faster, while preserving or even enhancing the rich information content of magnetic resonance images, says Facebook.

The key will be to train artificial neural networks to recognise the underlying structure of the images in order to fill in detail omitted from an accelerated scan."

Ah, right, what I want is for an ANN to invent information in a medical image. It's one thing to upscale textures in a game, but I can't see the use for this at all in a medical imaging device.

Re:DeepMind readies first commercial product by FeepingCreature on Monday April 01, 2019 @01:56AM (#19540870)
I don't care what it has to do, it can be drawing pentagrams in blood for all I care, as long as it has even a fraction better health outcomes to unassisted doctors I'm all for it.

The point here is making people healthy, not providing some sort of validated theoretically clean design. If the rcts say it works, bring on the deep learning voodoo.

(edit: I, for the record, want to trust in average accuracy.)

Re:DeepMind readies first commercial product by gdy on Monday April 01, 2019 @02:02AM (#19540898)
This is not how trust works
Re:DeepMind readies first commercial product by visarga on Monday April 01, 2019 @02:27AM (#19540994)
You mean it's not enough to trust in average accuracy, you want trust in specific instances of application?
Re:DeepMind readies first commercial product by nitrogen on Monday April 01, 2019 @05:53AM (#19541802)
Averages are not adequate descriptions of situations with bizarre outliers. An ML model that creates nonexistent detail in a medical image screams bizarre outliers.
Re:DeepMind readies first commercial product by FeepingCreature on Tuesday April 02, 2019 @06:28AM (#19552622)
Medical conditions are usually binary choices.
Re:DeepMind readies first commercial product by gdy on Tuesday April 02, 2019 @10:30AM (#19554439)
The article is about faster scanning, not making a diagnosis.
Re:DeepMind readies first commercial product by gdy on Tuesday April 02, 2019 @03:40AM (#19551878)
Well, imagine a doctor having a suspicion of some kind, sending a patient to get scanned and an AI ignoring an anomaly and substituting it with an interpolation of normal tissue. The doctor looks at the result and thinks he might have been wrong and says to the patient that he is fine.

Even if it happens more rarely than a doctor missing an anomaly on the scan, it doesn't feel right. A patient had a good chance to be diagnosed and he wasn't.

What's more, in case of normal MRI a patient or a doctor can ask another doctor to have a look, but in case of AI and the fast scan described in the article, the AI's interpretation of data is final, unless you go make another scan, this time in the traditional way.

Re:DeepMind readies first commercial product by FeepingCreature on Tuesday April 02, 2019 @06:29AM (#19552628)
I think this is more a problem with your feeling than the actual process.

You're focusing on the person who got falsely detected as healthy, and are ignoring the people who would be correctly detected as unhealthy. That's why the important question is expected outcomes.

I don't really care whether the doctor gets to be morally culpable or not. I'm not sure why I would ever care about something like that as a patient. If I get sick and die, the fact that a person was responsible rather than an algorithm gives me precisely zero comfort.

Re:DeepMind readies first commercial product by gdy on Tuesday April 02, 2019 @10:28AM (#19554410)
You falsely believe that it is a binary choice. Cost-cutting allowed by this fast and creative scanning is not the only way to help more patients.
Re:DeepMind readies first commercial product by pnloyd on Monday April 01, 2019 @12:37AM (#19540544)
I'd assume (hope) they mean to infer data and not invent it.
Re:DeepMind readies first commercial product by yeahitslikethat on Monday April 01, 2019 @02:35AM (#19541027)
It is clear to any mind of intelligence that you cannot give something less information and expect it to give you back better decisions.
Re:DeepMind readies first commercial product by overcode on Monday April 01, 2019 @05:27AM (#19541707)
Not only that you can - you have to do this to make better decisions. It's called focusing and everyone does it all the time.
Re:DeepMind readies first commercial product by deepnotderp on Monday April 01, 2019 @12:20AM (#19540470)
Idk. A lot of the applications of AI to compressed sensing are resulting in plausible but inaccurate data being filled in. This is fine for, say, video games, but not as great for medical imaging I'd guess.
Re:DeepMind readies first commercial product by skwb on Monday April 01, 2019 @12:44AM (#19540566)
I find it somewhat interesting that purely analytical techniques can be replicated using CNNs. Sorta goes to show that there's a sorta divergence in a purely EECS method vs some of the newer approaches that are able to integrate advanced signal processing with deep learning. Hopefully these hybrid approaches are able to advance the field. That said, MOST of the current publications out there on this topic don't seem to be pushing the field that far compared to conventional signal processing approaches.
Re:DeepMind readies first commercial product by eismcc on Sunday March 31, 2019 @11:23PM (#19540204)
This assumes that’s what medical companies want. If we get closer to full value based care, then yes.
Re:DeepMind readies first commercial product by xiphias2 on Monday April 01, 2019 @12:07AM (#19540412)
Which medical company doesn't want to move costs down?

You don't need high costs to charge a lot.

Re:DeepMind readies first commercial product by eismcc on Monday April 01, 2019 @12:22AM (#19540480)
If your business is making money on treating issues, preventing them isn’t aligned. Worst case, you neglect issues until they get to a point where you start making money. Legacy dialysis business is a good exsmple - they need ass-in-seats to justify clinics, so prevention isn’t as big a priority. Also, doctors often have shares in clinic revenue.
Re:DeepMind readies first commercial product by sergers on Monday April 01, 2019 @02:06AM (#19540911)
A friend of mind works for a software company that was utilizing Medq AI's brain bleed/hemorrhage detection Algorithm to identify urgent cases to be reviewed by a radiologist.

They were pitching it to a medical group, and they were like that's great for patient care but how does it cut costs to the organization...

Re:DeepMind readies first commercial product by xiphias2 on Monday April 01, 2019 @11:08AM (#19544052)
This makes sense, but Facebook / Google have enough resources to push their solutions onto the hospitals (give few years for free, maybe give the CEO some other fun stuff).
Re:DeepMind readies first commercial product by skwb on Monday April 01, 2019 @12:58AM (#19540626)
MRI manufacturers want to develop products that bring on new functionality and ability to justify their prices. A brand new 1.5T scanner easily costs several MIL to make, and for a clinic to purchase one will have to fulfill a business case to justify the purchase. For the clinics, acquiring new service lines (such as adding cardiac) is one such way to do so even in a more value based care setting.
Re:DeepMind readies first commercial product by ericcholis on Monday April 01, 2019 @12:35AM (#19540538)
I think this was already achieved in Switzerland without AI: https://www.dagensmedicin.se/artiklar/2018/11/20/en-mix-av-b...
Re:DeepMind readies first commercial product by vonnik on Monday April 01, 2019 @12:53AM (#19540608)
That's Sweden
Re:DeepMind readies first commercial product by skwb on Monday April 01, 2019 @01:05AM (#19540651)
I'll believe it if I see it at RSNA 2019.
Re:DeepMind readies first commercial product by shereadsthenews on Monday April 01, 2019 @12:25AM (#19540493)
Is that really the way to make the MRI faster? It seems like a time and motion problem. You lie in the machine for a long time but there's a lot of slack; the actual image process takes up barely any of the time. Remainder of time is spent waiting for things to whiz around and for the tech to click "next" on their console.
Re:DeepMind readies first commercial product by krapht on Monday April 01, 2019 @12:38AM (#19540546)
That's the problem, current MRI's are like taking high-def pictures in raw format 1 pixel at a time. There's loads of redundant information in the raw image encoding. If there was some way to only image the information we needed, the process could go faster.

Adding more "cameras" to our MRI so we can do parallel imaging is expensive.

Re:DeepMind readies first commercial product by skwb on Monday April 01, 2019 @12:48AM (#19540583)
Depends on what you're trying to image. Tbh, if it takes 15 mins vs 30 mins to get an image of your knee, it's maybe not really that much of a game changer.

From the perspective of a MRI purchaser (say clinic or hospital director), maybe you can squeeze 1 or 2 more cases in a day, but it seems like a marginal improvement overall. I think cardiac is clearly the exception where you bring on board potentially new functionality with quicker acquisition.