News for hackers. Stuff that matters.
Re:D6tflow: Python library for building data science workflows by L_226 on Saturday March 02, 2019 @12:01PM (#19289967)
This looks pretty interesting. I am currently implementing DS workflows that are essentially python classes to orchestrate R scripts. I'll have a closer look on Monday but if I can use it to handle rpy2 R format data I'll be happy
Re:D6tflow: Python library for building data science workflows by perturbation on Saturday March 02, 2019 @01:53PM (#19290526)
Are you using reticulate (https://github.com/rstudio/reticulate), or having Python spawn a new worker process for R?
Re:D6tflow: Python library for building data science workflows by L_226 on Saturday March 02, 2019 @03:56PM (#19291123)
We have python spawn processes for R.
Re:D6tflow: Python library for building data science workflows by altairiumblue on Saturday March 02, 2019 @12:12PM (#19290025)
I've been wondering about this - why would you pass data from one language to the other? If you already have existing R code, you can easily manage it through R itself.
Re:D6tflow: Python library for building data science workflows by L_226 on Saturday March 02, 2019 @12:30PM (#19290103)
I suppose we could have written an R API to serve ML results directly, but there are a lot of other auxiliary tasks needed like protobuf handling, AWS integration and some other business logic that means python makes more sense. The quickest way to make it work was to wrap the ML scripts (and they really are __scripts__, not even functions..) in python and just handle everything outside of R. I think eventually we will just re-implement the ML stuff in python anyway.