News for hackers. Stuff that matters.

A comment on Dataset of 13 Billion Clicks available for research.

Re:Dataset of 13 Billion Clicks available for research by nivla on Saturday January 19, 2013 @10:01PM (#5085814)
Oh boy, this is going to end badly. Remember when AOL released a huge anonymized dataset of their searches? People were still identified because naturally people searched for their own names, the names of their friends and families, local businesses, personal websites, etc.

This dataset is even worse since it includes both referral and the destination.

Keep in mind websites often put the usernames within the URL

Eg: http://www.facebook.com/Your.Name

http://www.reddit.com/user/USERNAME/

http://slashdot.org/~USERNAME

http://news.ycombinator.com/user?id=USERNAME

So no matter how much you think you have it anonymized, a person's browsing history could reveal a lot more than you think.

[ Parent | on HN ]
Re:Dataset of 13 Billion Clicks available for research by politician on Sunday January 20, 2013 @01:52AM (#5086238)
The data doesn't seem to contain enough information to cluster unrelated URLs (e.g. by User Agent). Each record contains 1) a timestamp, 2) the requested URL, 3) the referring URL, 4) a boolean classification of the user agent (browser or bot), 5) a boolean flag for whether the request was generated inside or outside IU.

Although you could reconstruct who's looking at Facebook from inside IU, it'd be difficult to further associate them with some other website like Hacker News. On the other hand, the timestamps do leak some information which could potentially be used to identify patterns of activity.

[ Parent | on HN ]
Re:Dataset of 13 Billion Clicks available for research by kordless on Sunday January 20, 2013 @05:59PM (#5088644)
You've seen that bit where the user-agent is fairly unique to a particular user, right?

https://www.eff.org/deeplinks/2010/01/tracking-by-user-agent

[ Parent | on HN ]
Re:Dataset of 13 Billion Clicks available for research by politician on Monday January 21, 2013 @12:17AM (#5089993)
Yes, but they aren't sharing the user agent. They're just sharing a boolean derived from the user agent which says whether they think it was a bot or a browser.
[ Parent | on HN ]