Re:Hamilton: A Microframework for Dataframe Generation
by elijahbenizzy on Thursday December 02, 2021 @12:10PM
(#29418988)
Author here -- that's awesome! Great minds think alike. Always interested in use-cases (that both fit and don't) -- feel free to open up an issue with something representative/what you want and I'm happy to contribute my opinion.
I had a very similar idea recently (representing "big" preprocessing pipelines in Python as DAGs), but for research-scale projects. Funnily enough, I was also motivated by a time series project that has resulted in several thousand lines of gnarly preprocessing code, and growing - mostly in Pandas.
I'm not sure Hamilton would totally work for our use case, but I'll be following closely either way. Thank you for open sourcing this!