Re:Machine Learning in Production (CMU Course)
by belter on Tuesday January 28, 2025 @07:14AM
(#42851462)
This seems to have very little on Data Quality and it is on Chapter 16...How much practical experience in Industry do the authors have? Because 90% of your time will be spent on Data Quality and Data Cleansing...
[ Parent
| on HN ]
Re:Machine Learning in Production (CMU Course)
by szvsw on Tuesday January 28, 2025 @10:10AM
(#42853178)
Arguably that’s a separate (obviously critical) concern. I think it’s worth it to abstract that away as just a step that exists in the pipeline with its own set of concerns/challenges/methods etc that really requires its own deeper study to do well.
For instance, my ML work is almost entirely in the context of engineering simulation regression/surrogate development, where data quality/cleaning is almost no issue at all - all of the work is on the dataset generation side and on the model selection/training/deployment side.
Every job is different!
Re:Machine Learning in Production (CMU Course)
by bigmutant on Tuesday January 28, 2025 @02:57PM
(#42857161)
Agree, Data Quality in-the-wild is a huge concern. I've led efforts to establish Lineage/Quality in large orgs and doing this after-the-fact is a massive undertaking. Having this "up-front" before all the data pipelines (origination, transformation, pre-processing) calcify saves a lot of headache down the road.