Re:How I came to love big data
by etrain on Friday November 09, 2012 @12:07PM
(#4763673)
The tails.
[ Parent
| on HN ]
Re:How I came to love big data
by yummyfajitas on Friday November 09, 2012 @12:22PM
(#4763740)
You also often need to scan the whole data set in order to sample properly.
Suppose you are interested in user behavior. You want sample(group_by(user_ident, all_user_events)), not group_by(user_ident, sample(all_user_events)). This involves running a group_by on the full data set.