News for hackers. Stuff that matters.
Re:Scikit-Learn Version 1.0 by civilized on Tuesday September 14, 2021 @09:04AM (#28524118)
As a working data scientist I'd say it's completely the opposite: a good tabular data manipulation package is the single most valuable tool in my tool box. And R's packages (either data.table or dplyr) are definitely way better than pandas. There's no comparison.

I would be hard-pressed to find a working data scientist whose definition of data science is "that thing you do with sklearn, Deep Learning and Numpy".

Re:Scikit-Learn Version 1.0 by pletnes on Tuesday September 14, 2021 @09:10AM (#28524204)
Tabular data is great for many usecases, but saying that image, audio, and video analysis is not data science seems like a weird variant of gatekeeping to me.
Re:Scikit-Learn Version 1.0 by disgruntledphd2 on Tuesday September 14, 2021 @10:10AM (#28525084)
> Tabular data is great for many usecases, but saying that image, audio, and video analysis is not data science seems like a weird variant of gatekeeping to me.

Most problems are mostly tabular, IME.

I completely agree that text, images and video are much, much better handled by Python (that's why I use and know both).

Re:Scikit-Learn Version 1.0 by civilized on Tuesday September 14, 2021 @09:16AM (#28524287)
Of course, my post doesn't imply such a silly statement.
Re:Scikit-Learn Version 1.0 by jstx1 on Tuesday September 14, 2021 @09:09AM (#28524189)
> "Data science is that thing where you do sklearn, Deep Learning and Numpy" is not a working data scientist's perspective.

It could be. It's such a broad job title and it looks so different across different companies and teams that the main tool for one data scientist might be something that another data scientist never has to touch. Different data science jobs prioritise different tools, that's all.

Re:Scikit-Learn Version 1.0 by civilized on Tuesday September 14, 2021 @09:18AM (#28524328)
Right, so defining data science as 90% sklearn+DL+numpy is just as silly as saying that it's 90% table manipulation. That's exactly my point.

Still, if anyone here has managed to find a data science job in which tabular data management is not a sizable piece of what you do, I'd like to know some details!

Re:Scikit-Learn Version 1.0 by mrtranscendence on Tuesday September 14, 2021 @09:48AM (#28524744)
I imagine there are data scientists who operate primarily on unstructured rather than tabular data. Part of my current job involves stuff like text classification, and it's not that difficult to imagine someone for whom that's a more sizable proportion of their day-to-day.

Still, my suspicion -- at least from my corner of data science -- is that such individuals are rare, and that most data scientists do make use of tabular data more often than not.

Re:Scikit-Learn Version 1.0 by civilized on Tuesday September 14, 2021 @10:08AM (#28525048)
I totally get what you mean - I would suspect that when you work with unstructured data, tabular data manipulation is maybe 20-40% of what you do, and when you work with structured data, it's more like 60-80%.
Re:Scikit-Learn Version 1.0 by _Wintermute on Tuesday September 14, 2021 @11:52AM (#28526636)
I worked as a datascientist for a couple of years and tabular data was a very small part of my job. I spent far more time with image-analysis and JSON, both of which I found R sucks at.