News for hackers. Stuff that matters.
Re:The R learning curve by jasonpbecker on Thursday February 07, 2013 @09:53PM (#5186031)
plyr, while having awesome syntax, is really slow for anything beyond even the most modest dataset. Try using it with 50k rows, not to mention true "big data".

data.table is much faster. It's an extension to data.frames that adds some additional constrains/rules that allows for much faster operations including aggregating, subsetting, and merging data.

Cleaning data, however, does not have a steep learning curve or high difficulty level in R-- it has a steep learning curve and high difficulty level period. Implementing good procedures for data munging is 80% of the job.

Re:The R learning curve by EzGraphs on Thursday February 07, 2013 @10:00PM (#5186050)
Yeah - for the small data sets, Ruby or Python is often easier easier to use to whip a data set into a form that can be simply slurped into R as a dataframe.
Re:The R learning curve by mogrim on Friday February 08, 2013 @04:24AM (#5187022)
Has to be said I cheated a bit on the Coursera course, I used Ruby to clean up some of the data. Seemed a lot easier, and it's a lot more likely I'd use some kind of scripting language (be it Perl, Ruby, Grep, ...) to do a first clean up than head directly to R to do the job.
Re:The R learning curve by aheilbut on Thursday February 07, 2013 @11:19PM (#5186261)
With pandas / statsmodels / patsy, it's getting easier to stick with python for everything.
Re:The R learning curve by GrumpySimon on Friday February 08, 2013 @12:02AM (#5186378)
Don't forget Bokeh[1] - a replacement for ggplot2

[1] https://github.com/ContinuumIO/Bokeh

Re:The R learning curve by banachtarski on Friday February 08, 2013 @12:26AM (#5186450)
This is a good point. Usually, I use haskell or something to do large scale transformations prior to a plot or something. However, I still use plyr from time to time to do quick one offs on smaller samples.
Re:The R learning curve by hadley on Friday February 08, 2013 @07:58AM (#5187596)
With 50,000 rows plyr should be fine, and I've tested it up to 1,000,000 and it's reasonably. The problem is more that it doesn't scale terribly well in the number of groups.