Re:Why software engineering processes and tools don’t work f...
by PaulHoule on Wednesday October 14, 2020 @09:00PM
(#24783607)
I have scars from the battles that the author describes and I don't recognize the author as a fellow veteran.
The main problem is that "data scientists" don't follow procedures at all in their work. If they applied conventional software engineering so far as it applies they would do a lot better.(e.g. use version controls but don't check the database password in, think carefully what configuration goes with the hardware, the "model" in general, a particular instance of the "model", etc.)
ML projects get you in trouble in other areas.
For one thing, an ML project might involve a step that takes 24 hours or more to train a model. This means you have to schedule in terms of calendar time (e.g. can't start task A until task B is complete) as opposed to punch clock time, story points, or whatever passes for reality in your flavor of agile.
Related to that is automated testing. Normal software "works or doesn't work" but ML software is expected to get it wrong a certain fraction of the time so the question of "Is version x.y.z good to ship?" is fundamentally different in nature with ordinary software.
It irks me particularly that he ends his circle with "harvest insights" when any real game theorist knows it is about taking actions based on information -- e.g. why winners at gambling and trading use the Kelly Criterion.
The main problem is that "data scientists" don't follow procedures at all in their work. If they applied conventional software engineering so far as it applies they would do a lot better.(e.g. use version controls but don't check the database password in, think carefully what configuration goes with the hardware, the "model" in general, a particular instance of the "model", etc.)
ML projects get you in trouble in other areas.
For one thing, an ML project might involve a step that takes 24 hours or more to train a model. This means you have to schedule in terms of calendar time (e.g. can't start task A until task B is complete) as opposed to punch clock time, story points, or whatever passes for reality in your flavor of agile.
Related to that is automated testing. Normal software "works or doesn't work" but ML software is expected to get it wrong a certain fraction of the time so the question of "Is version x.y.z good to ship?" is fundamentally different in nature with ordinary software.
It irks me particularly that he ends his circle with "harvest insights" when any real game theorist knows it is about taking actions based on information -- e.g. why winners at gambling and trading use the Kelly Criterion.