This is changing (shameless plug for my company http://www.pulmolux.co.uk) but medicine moves slowly preferring evolution over revolution. Having said that, I certainly detect that the scanner manufacturers (GE, Philips, Siemens, etc) having reached saturation with radiologists now have a thirst for disruption and see the referring physician as the next customer. MRI cardiology being something of an example.
I work on radiology image segmentation also. And I agree it is solvable with machine learning. But even if software could do a job of a radiologist, it wouldn't replace one any more than your ekg reading program replaced cardiologists.
I don't think the radiologist is going anywhere soon but the role is changing. Radiologists are increasingly having to deal with more and more derived information. They need to understand the algorithms being used as well as the biology, anatomy, physiology and disease being investigated. I can see a time when algorithmic specialists become a regular part of their multidisciplinary team.
I can see how my phrasing was confusing, but I didn't mean to suggest that Ct scans are used to look for brain tumors. My work segmenting brain scans was not tumor-related, just gray/white matter segmentation.
I suspect the first major hospital-facing implementations of machine learning will be in radiology, e.g.: http://suzukilab.uchicago.edu/, which has been diagnosing cancerous tumors in CT scans with neural networks since before it was cool (one reason you won't see the terms 'deep learning' in the literature since they were originally just 3-layer networks, before the term was even coined). IIRC it outperformed the average radiologist.
I wonder if the label problem could be less difficult for some low-hanging fruit. The CT scan neural network required something like 40k labeled scans from a radiologist, but it could come for free: many yes/no disease detections will eventually be resolved by human labeling anyway by your doctor. If you had access, say, to every CT scan taken, and electronic health records for the patients, your labeling is noisy and biased but at least at massive scale. The problem is (legitimately) restricted access to health records in the US. Maybe some European countries have better data access?
And the implementation problem will eventually disappear. I remember talking with a radiologist years ago, who remarked "some people in my field have no idea it's about to disappear". I'm not so sure there will be no more radiologists, but their role will definitely change. Hospitals would be okay with this, actually, since radiologists are expensive. Eventually radiology scans will probably be like ordering blood tests, where fewer and fewer MD's are required.