News for hackers. Stuff that matters.
Re:R Passes SAS, but Python Leaves Them Both Behind by fnord123 on Sunday March 05, 2017 @01:16PM (#13797447)
If you work in a team mixing data engineers and data scientists then python has been a superior choice for a decade as the different team members are all using the same language to build the platform and to use it.

We've had a lot of success weaning people of R and MATLAB in finance (so replace "data scientist" with "quant"). Of course if you work in a field that doesn't have the libraries and can't build them in house then your mileage will certainly vary.

Re:R Passes SAS, but Python Leaves Them Both Behind by rm999 on Sunday March 05, 2017 @03:50PM (#13798524)
This is a great point. Python has been a solid data engineer/pipeline language for years now. It makes it easy for data scientists and data engineers (or people who wear both hats) to work in the same environment and codebase.
Re:R Passes SAS, but Python Leaves Them Both Behind by est on Monday March 06, 2017 @12:11AM (#13800832)
The only area of un-touched by Python is web frontends and mobile apps.

Hope Web assembly would change that.

Re:R Passes SAS, but Python Leaves Them Both Behind by thearn4 on Sunday March 05, 2017 @05:22PM (#13799161)
Off-topic, but what is a "data engineer"?
Re:R Passes SAS, but Python Leaves Them Both Behind by nl on Sunday March 05, 2017 @11:47PM (#13800763)
In my team, "data engineer" refers to someone who deals with the ETL process, but also some feature engineering (ie, designing and extracting features. There's a lot of cross over here with the work Data Scientists do, but often the tools are different: for example, in the Python world the data engineer will use Pycharms while the data scientist will use Jupyter.
Re:R Passes SAS, but Python Leaves Them Both Behind by nerdponx on Sunday March 05, 2017 @05:34PM (#13799237)
I'm sure someone can come along with a better description, but it's kinda like devops/sysadmin but specifically for data storage and access.

You could also look it up...

Re:R Passes SAS, but Python Leaves Them Both Behind by sdenton4 on Sunday March 05, 2017 @11:26PM (#13800696)
At some point, people realized that 90% of data science work was building pipes and keeping them clean. And then data engineering was invented so that the data scientists wouldn't have to get their hands as dirty. :)
Re:R Passes SAS, but Python Leaves Them Both Behind by fnord123 on Monday March 06, 2017 @04:55AM (#13801695)
A snappier version which gives the flavour: Data scientists use {tensor flow, scikit-learn, parquet, hadoop, spark, scrapers, etc}. Data engineers write {tensor flow, scikit-learn, parquet, hadoop, spark, scrapers, etc}.
Re:R Passes SAS, but Python Leaves Them Both Behind by xapata on Sunday March 05, 2017 @09:24PM (#13800274)
I'd include database engineer in that bin. We used to call people DBAs but that seems a bit limiting now.
Re:R Passes SAS, but Python Leaves Them Both Behind by mehaveaccount on Sunday March 05, 2017 @06:49PM (#13799646)
That's a good description for a data engineer from my end.