On the machine learning side, there is very little in the way of inference. Most of machine learning is an attempt to solve the statistical problem of predicting some value Y (e.g. a diagnosis) given a feature vector X.
To obtain data from medical texts (even with human assistance) and use this to form diagnoses of actual patients, would require solving both these problems. We would have to parse and represent the complex logical relationships that are expressed in academic papers. We would then have to have machine learning methods that can work with complex data (not just feature vectors) and combine inference with statistical learning.
And isn't doctors and insurance companies being impressed with results are another hint that watson has those capabilities(or is a very big fraud, something IBM wouldn't riskm) ?
Unlike with jeopardy, you do not have a huge well organized catalog of information lovingly curated by trivia worshiping nerds. It is a ridicules stumbling block but there you have it. Health records are not normalized and you can't touch them anyway, research is closer to human readable prose and filled with jargon - good luck developing NLP for it if you're "just a programmer". Hell, look at their textbooks and the way they study and are trained, that screams for computerized improvement and is still a tall order.
Watson is able to parse natural language and pull out possibly important information. It then applies some very state-of-the-art machine learning to derive the most likely contextual meaning of that information. That contextual meaning can be cross-referenced with billions of other pieces of weighted information to derive further insights.
I come from a family full of doctors. They tend to be smart people, but as Jeopardy showed, even the best player on earth is no match for a machine trained at that discipline.
Once we start adding in genetic information, that will allow machines like Watson to customize diagnoses and treatments based upon your specific genetic profile.
The way they are implemented is by people entering data in an orderly fashion, both those who built the systems ,and doctors . This somewhat limits the system, both from breadth of research and data entry time at the doctor.
But even with those limitations , those systems offer far better diagnoses , especially in the rare cases,than you average and even great doctor.
This isn't after-the-singularity type science fiction where we're adding assumption upon assumption to several levels beyond what is possible.
This is applying a technology (that's already proven at performing similar complex deductive analysis over massive quantities of data) to a new discipline. Any assumptions about what we can do in the new discipline with the current technology are minimal and extremely likely overcomable.
Even if Watson isn't quite there, the way is pretty clear now. Linear improvements in the Watson approach will yield results.