News for hackers. Stuff that matters.
Re:Tips for Shipping Data Products Fast by boringds on Thursday March 04, 2021 @10:07AM (#26343301)
I think you nailed it. Often companies and exec want ML but don't have the basics: robust ETL pipeline, clean data, solid analytics foundation (dashboards, automated reporting, etc.). These appear boring to most but they will be the difference between a useless ML department who can't ship anything to production and a successful one that builds on top of the aforementioned foundations.

In addition I believe it's time we drop the data science term. It's an umbrella of different roles ranging from data engineer to DL researcher. Companies need to identify what they REALLY need and not go for the shiny PhD in ML.

The emergence of analytics engineering is the perfect example of this shift towards creating robust data pipeline first and enabling "data scientists" to do so.

I wrote a blog post about it yesterday, I don't want to post it here and self-promote too much, so check it in my profile if you want to.

Re:Tips for Shipping Data Products Fast by PebblesRox on Thursday March 04, 2021 @04:51PM (#26348925)
Here's boringds's post, for anyone else who's curious: https://boringdatascience.com/post/data-science-is-dead-long...
Re:Tips for Shipping Data Products Fast by nerdponx on Thursday March 04, 2021 @05:48PM (#26349596)
I love the idea of "boring data science." I will steal that term for my own use.
Re:Tips for Shipping Data Products Fast by EricMausler on Thursday March 04, 2021 @10:59AM (#26344007)
Who do you think has the best 3rd party solutions for data cleaning?
Re:Tips for Shipping Data Products Fast by klmadfejno on Thursday March 04, 2021 @11:11AM (#26344204)
Data cleaning is domain specific. Hire someone to do it and accumulate wisdom over time.
Re:Tips for Shipping Data Products Fast by boringds on Friday March 05, 2021 @03:33AM (#26354375)
For data cleaning I swear by dbt (https://www.getdbt.com/). It's such a powerful tool that you can put in the hand of anyone with SQL knowledge. It allows us to develop clean pipelines, document, and test our data easily. It's also free and the team working there is amazing.