Re:Kedro – Creating reproducible, maintainable and modular d...
by bshipp on Saturday January 22, 2022 @02:10PM
(#30039809)
It might be opinionated but it's no less accurate.
Data science is many researchers first foray into programming in any language and, speaking from my own path, was a relatively self-directed discipline in the early days.
I often longed for a 'standard approach' but many of the projects that suggested them were packed full of boilerplate that made code even harder to read and write, especially if your background wasn't heavily into OOP.
Re:Kedro – Creating reproducible, maintainable and modular d...
by waylonwalker on Sunday January 23, 2022 @10:00AM
(#30046580)
I feel ya. Before using kedro I was using a version of the cookiecutter for data science template that I modified to have something kind of like their data catalog. Before that it was so wild west, every project was its own unique thing.
Re:Kedro – Creating reproducible, maintainable and modular d...
by waylonwalker on Sunday January 23, 2022 @09:58AM
(#30046564)
Maybe a better word is that it comes with an opinionated template? The core parts of Kedro can all be used individually as you wish. Only want a DAG, you can use just the pipeline, Want a catalog setup so you can store intermediate steps, use the Catalog. You can even make kedro project using all the features in a single file with surprisingly little boilerplate.
I mean I guess it's "opinionated" but having worked in this field for some time, ML code and pipelines are often laughably immature, stuck with spit, glue and sellotape. Kedro forces people into a bit of a straight jacket for sure, but man does it make things more readable.
Plus points, it's very extensible so if there's a design element you're missing or one of its opinions you just can't abide by, then you can extend it. I think it deserves more exposure.