News for hackers. Stuff that matters.
Re:How I came to love big data by etrain on Friday November 09, 2012 @12:07PM (#4763673)
The tails.
Re:How I came to love big data by yummyfajitas on Friday November 09, 2012 @12:22PM (#4763740)
You also often need to scan the whole data set in order to sample properly.

Suppose you are interested in user behavior. You want sample(group_by(user_ident, all_user_events)), not group_by(user_ident, sample(all_user_events)). This involves running a group_by on the full data set.

Re:How I came to love big data by tel on Friday November 09, 2012 @01:11PM (#4763975)
Or counting the number of events for each user as they come in.