News for hackers. Stuff that matters.
Re:What Data Scientists Really Do, According to Data Scientists by appleiigs on Saturday August 25, 2018 @09:34PM (#17843695)
“It has been a common trope that 80% of a data scientist’s valuable time is spent simply finding, cleaning, and organizing data, leaving only 20% to actually perform analysis.”

Is it really trope? For my experience I almost think collecting data is >80%.

Re:What Data Scientists Really Do, According to Data Scientists by gaius on Sunday August 26, 2018 @04:04AM (#17844854)
80% of time is spent on getting and cleaning the data, 20% of time is spent preparing and delivering reports and presentations.

The super-cool ML stuff that attracts people to the field in the first place, accounts for little more than a rounding error in how the time is really spent

Re:What Data Scientists Really Do, According to Data Scientists by visarga on Sunday August 26, 2018 @12:09AM (#17844204)
I get to spend 90% collecting, cleaning and tagging data.
Re:What Data Scientists Really Do, According to Data Scientists by pbhjpbhj on Sunday August 26, 2018 @09:39AM (#17845869)
Sounds like poor role definition? You don't have your laboratory scientist cleaning lab equipment or ordering reactants.
Re:What Data Scientists Really Do, According to Data Scientists by narens on Sunday August 26, 2018 @01:25AM (#17844419)
https://dremio.com tries to solve this exact problem. It’s open source too.
Re:What Data Scientists Really Do, According to Data Scientists by FuriouslyAdrift on Tuesday August 28, 2018 @08:50AM (#17858381)
so I checked out the website and I still don't get what Dremio actually is...

I know what I need which is a wide open easy to use (think CEO's personal assistant who has zero background in anything other than basic business processes easy) visual ETL tool.

We have found some decent VERY expensive and VERY user unfriendly tools but nothing cheaper than just hiring a person to custom handle each batch.

Basically...we're stuck with Cognos, et al.