Re:IBM Watson correctly diagnoses a form of leukemia
by fennecfoxen on Sunday August 07, 2016 @08:20AM
(#12241883)
You don't need 100% new data to get an idea of performance. It's meaningful (indeed, common) to select some arbitrary percentage of the real-world data as a "training set" then measure the performance of the algorithm on the rest. It's not proof against overfitting, of course, but it helps.
[ Parent
| on HN ]
Re:IBM Watson correctly diagnoses a form of leukemia
by apathy on Sunday August 07, 2016 @04:08PM
(#12243421)
right, but there's a reason we run clinical trials before accepting that something is superior to standard of care. It is really, really hard to manipulate a preregistered clinical trial (not that people don't try) and incredibly easy to manipulate train/test/validate results.