News for hackers. Stuff that matters.
Re:Will R Work on Apple Silicon? by minimaxir on Wednesday November 11, 2020 @11:05AM (#25059458)
As a data scientist who is proficient in both Python and R ecosystems, in my opinion R/tidyverse is substantially better for ad hoc EDA and data visualization.

However, Python is better for nearly everything else in the field (namely, working with nontabular data, external APIs, deep learning, and productionization).

It's about knowing which tool to use.

Re:Will R Work on Apple Silicon? by curiousgal on Wednesday November 11, 2020 @12:14PM (#25060277)
You forgot time series analysis where Python is years behind R. Robust regression methods too. But most of all, Shiny! Python's Dash for creating interactive data web apps is absolutely horrible compared to Shiny.

Basically if you were to go down any unbeaten path when it comes to statistical models, you're better off using R. But if your main goal is pushing to something prod, then you're better off with Python. The only exception being Shiny, they've put a lot of effort into making it production-ready.

Re:Will R Work on Apple Silicon? by jerjerjer on Wednesday November 11, 2020 @12:44PM (#25060655)
> You forgot time series analysis where Python is years behind R

What does R offer?

In Python there's SARIMAX and Prophet, interested in what R has to offer.

Also interested in a decent Grid Search for time series.

Re:Will R Work on Apple Silicon? by minimaxir on Wednesday November 11, 2020 @01:23PM (#25061127)
R also has prophet, with some native ggplot2 integrations. https://facebook.github.io/prophet/docs/quick_start.html#r-a...
Re:Will R Work on Apple Silicon? by jerjerjer on Wednesday November 11, 2020 @02:30PM (#25061936)
Thanks, but I'm more interested in what R has on time series, which Python does not.

Just checked Prophet source code and it looks like there are actually 2 implementations in the same repo: one for R and one for Python. I thought it would have been the usual C/C++ implementation with 2 bindings. I wonder why they chose to develop it this way.

Re:Will R Work on Apple Silicon? by dash2 on Wednesday November 11, 2020 @02:17PM (#25061793)
R has the builtin ts library, and also see this Task View with literally hundreds of packages: https://cran.r-project.org/web/views/TimeSeries.html
Re:Will R Work on Apple Silicon? by mr_toad on Wednesday November 11, 2020 @05:40PM (#25063890)
Re:Will R Work on Apple Silicon? by bllguo on Wednesday November 11, 2020 @03:49PM (#25062818)
tidyverts and fable are very nice, though I'd also be interested in seeing what others have to say, as I don't do much timeseries analysis. Prophet pretty much covers all my use cases thus far
Re:Will R Work on Apple Silicon? by thenipper on Wednesday November 11, 2020 @01:21PM (#25061105)
You can use Prophet in R for sure
Re:Will R Work on Apple Silicon? by data_ders on Wednesday November 11, 2020 @07:59PM (#25065040)
Hyndman is our king!
Re:Will R Work on Apple Silicon? by omarhaneef on Wednesday November 11, 2020 @11:15AM (#25059560)
Ditto (not that I am proficient, but my experience matches).

However, because the rest is easier in python, and my mental gears grind when I switch from one to the other, I end up using Python for the adhoc EDA and viz, and with Spyder, it is a pretty decent experience.

Re:Will R Work on Apple Silicon? by jftuga on Wednesday November 11, 2020 @05:45PM (#25063917)
R is 7th on this list:

https://pypl.github.io/PYPL.html

Re:Will R Work on Apple Silicon? by disgruntledphd2 on Wednesday November 11, 2020 @11:33AM (#25059772)
> (namely, working with nontabular data, external APIs, deep learning, and productionization).

I agree with all of that except for productionisation. I would have agreed before dealing with issues around getting consistent versions of python + libraries to run.

The issues I see with Python are as follows:

- pip doesn't actually check to make sure your dependencies are compatible, which causes real problems with numpy et al

- conda isn't available by default, and running it on remote boxes is non-trivial (I spent a whole week figuring out how to get it running in a remote non-login shell).

- This makes it really, really difficult to actually get a standard set of libraries to depend upon, which is really important for production.

R, on the other hand, actually resolves dependencies in its package manager, and the R CMD BUILD for packages, while super annoying helps you produce (more) portable code (did you know that conda doesn't provide cross-platform yml files unless invoked specifically?).

In terms of handing it over to engineering/non data science people though, Python is much much much better.

tl;dr Python's an ace language with a terrible production story.

Re:Will R Work on Apple Silicon? by hobofromabroad on Wednesday November 11, 2020 @11:56AM (#25060063)
I agree and disagree. We deploy our models via Cloud Foundry which has support for Anaconda.

Model building is done in AWS with access to Anaconda.

Usually we have an environment.yml for the REST API and one for model building.

This makes modeling -> deployment cycle fairly easy, if not perfect.

You can also use pip and env, but you have to make sure that all important dependencies are specified sufficiently specific. But that's also the case for Anaconda. (For instance, we had a problem in the API with a x.x.y release of greenlet or gevent since we only specified x.x)

For R, well use packrat. R IMHO has the problem of many different algorithms with different APIs. Yes, there are tools like caret, but 'you' will run into problems with the underlying implementations eventually. sklearn makes things easier here, at least most of the time.

I would also prefer R for EDA. But I don't like splitting eda and modeling that way, since there can be subtle differences in how data is read which can lead to hard to find problems later on. (Yes, you could use something like feather)

I also thing that tooling for python is much nicer, pytest, black, VSCode python integration just seem more mature.

Re:Will R Work on Apple Silicon? by ekianjo on Wednesday November 11, 2020 @11:40AM (#25059868)
> R, on the other hand, actually resolves dependencies in its package manager

And you have packages like renv which also help isolate specific versions of packages to make portable environments even more reliable.

Re:Will R Work on Apple Silicon? by klmr on Thursday November 12, 2020 @04:16AM (#25067987)
‘renv’ (only) does more or less what pyenv does. Contrary to what the parent comment says, R doesn’t actually do any dependency resolution at all, and the official package repository (CRAN) doesn’t even archive many old versions (though MRAN does).

I strongly prefer R for data science, but its dependency management story is poor, even compared to Python’s (which, in turn, is poor compared to Rust/Ruby/…).

Re:Will R Work on Apple Silicon? by disgruntledphd2 on Thursday November 12, 2020 @09:27AM (#25070207)
That's simply not true. R doesn't store old versions, which is actually brilliant because your code breaks when your dependencies rot.

Python will silently upgrade numpy as a transitive dependency and break everything, which is much worse. MRAN also has daily snapshots which is normally how i handle stuff that will never be updated.

I also specified building an R package which does handle dependencies versus the python equivalent which does not.

I'm not saying R is good, I'm just saying Python is way worse.

Re:Will R Work on Apple Silicon? by guitarbill on Thursday November 12, 2020 @12:26PM (#25072126)
> even compared to Python’s (which, in turn, is poor compared to Rust/Ruby/…)

Compared to Rust? Sure. Compared to Ruby? Maybe in the way that a lockfile isn't automatically generated when using pip.

Hating on Python's dependency management is a meme at this point. You could do a lot worse than the current pip + venv, and upgrading to something like poetry or pipenv is pretty painless. I'm pretty sure 99% of problems occur because people don't pin stuff.

Re:Will R Work on Apple Silicon? by laichzeit0 on Wednesday November 11, 2020 @11:48PM (#25066553)
And what libraries do you use in R to generate a REST api that has Swagger 3 documentation? Authentication with JWT tokens? Monitoring, e.g. ApplicationInsights?
Re:Will R Work on Apple Silicon? by disgruntledphd2 on Thursday November 12, 2020 @02:59AM (#25067559)
For that, R would not be the right choice. I stand by my comments on how difficult it is to productionise python ML applications.

I think all of those things you mentioned are JVM stuff, right? There's a version of R called renjin that could be used in that scenario.

Don't get me wrong, I'd love if this was better in python but right now it is far more difficult than it needs to be.

Re:Will R Work on Apple Silicon? by laichzeit0 on Thursday November 12, 2020 @03:24AM (#25067696)
Nope, none of it is JVM stuff. It's pretty standard stuff if you want to ship an API into a production environment and expect other developers/services to interact with your model. How do you know your model is failing/slow to serve requests? You need monitoring/logging. How do add security? I'm talking API security, like JWT tokens with scopes and claims.

Maybe we mean different things by "productionising ML applications" but building a docker container with an R runtime and the correct package versions is not all, or even half, of what's required for production.

Re:Will R Work on Apple Silicon? by pbowyer on Thursday November 12, 2020 @03:59AM (#25067876)
Why would you have any of this tightly coupled to your model?

Set up a separate API gateway, which covers all your points (REST endpoints, monitoring, security) - there's plenty of off-the-shelf options. Route authenticated requests to the backend that runs your model.

Re:Will R Work on Apple Silicon? by disgruntledphd2 on Thursday November 12, 2020 @09:23AM (#25070152)
Depends on your model. Mine score users daily, so I don't need to worry about building an API.

Logging is pretty available in both (though better in Python to be fair).

I don't really see how building my model in Python would make it easier to add this API functionality either, so it's a bit irrelevant. Like my docker container (which appears to be almost essential in Python but nice in R) can call predict in any language, and then pass through to the API using the tools noted above.