Re:Four lessons from a year building tools for machine learning
by Imnimo on Friday July 16, 2021 @07:44PM
(#27862418)
>In just a few hours the lawyers had trained a model that provided the outcome of all 80,000 judgements without needing the input of a data scientist at all.
If this is meant to imply it predicted them all correctly, that rings alarm bells to me. 100% accuracy is much more likely to mean something is wrong (label leakage?) than it is to mean you have an amazing model.
Re:Four lessons from a year building tools for machine learning
by razcle on Friday July 16, 2021 @07:50PM
(#27862459)
No, not 100% accuracy. I've left out details for the sake of brevity but with a precision and recall high enough for the team to be able to answer the questions they cared about.
If this is meant to imply it predicted them all correctly, that rings alarm bells to me. 100% accuracy is much more likely to mean something is wrong (label leakage?) than it is to mean you have an amazing model.