Re:Python vs. R: The battle for data scientist mind share
by padthai on Saturday April 08, 2017 @10:21AM
(#14067040)
All these articles always say that Python has better preprocessing compared with R. Where? I find tidyverse/data.table much more elegant than pandas, scipy et al. The only thing that I like more in Python it is how it handles streams/generators.
[ Parent
| on HN ]
Re:Python vs. R: The battle for data scientist mind share
by laGrenouille on Saturday April 08, 2017 @06:17PM
(#14069368)
I use both R and Python extensively, and also was confused for a long time why the conventional wisdom is that one of Python's strengths is a data cleaning and preprocessing. I too always found data cleaning to actually be one of R's strongest features. My hypothesis: I think there is an inconsistency between what is meant by "cleaning/processing" between stats- and CS-oriented people. To the latter, it often just means turning unstructured or tree-like data (JSON/XML) into a tabular format. As a statistician, I think of it being a much larger set of operations; specifically those supported by packages such as dplyr, tidyr, and stringr... I'm not sure if that interpretation is correct, but Python does excel in the XML/JSON => CSV step and R is great at all the processing that happens after that.
Re:Python vs. R: The battle for data scientist mind share
by itschekkers on Saturday April 08, 2017 @11:52AM
(#14067492)
Ya, completely agree with this (having also used both extensively). dplyr can also connect to remote SQL server so data don't have to be local. Maybe pandas does this now too but in my experience SQL connections were generally more painful in python
Re:Python vs. R: The battle for data scientist mind share
by Analog24 on Saturday April 08, 2017 @12:14PM
(#14067613)
This is what SQLAlchemy is for. It's one of the best ORMs out there and makes interacting with a SQL server a breeze.
Re:Python vs. R: The battle for data scientist mind share
by __s on Saturday April 08, 2017 @11:06AM
(#14067255)
The last time I preprocessed with Python I only used csvreader. Client had sent a tree structure as two column table, so in one pass I outputted SQL for the recursive table with a bit of if-second-column-empty-then-figure-out-depth-of-first-column logic
Maybe R would work, but I'm familiar with Python. So Python stayed out of my way