News for hackers. Stuff that matters.
Re:FiveThirtyEight has a GitHub repo with story-related data... by danso on Tuesday May 31, 2016 @11:44PM (#11811932)
Sorry to seemingly play a game of one-upsmanship, as 538's repo is wonderful, but BuzzFeed also posts its data and stories online on their BuzzFeedNews Github account...however, what I really like about their repo setup is that they have a repo for every project...and then a separate repo that has a tabular listing of repo, date, description, and story link * ...IMO, they have the best setup (in terms of discovery) among the news nerds:

https://github.com/BuzzFeedNews/everything

- * edit: Oops, I forgot that 538's data repo also has a readable table listing below the fold...I suppose having separate repos per dataset vs one main repo is not a clear advantage depending on the user.

They also list their standalone datasets and tools, e.g. their standardized H-2 certification data [1], for which they've probably used in their many H-2 visa stories, and twick [2], the tool they use to quickly fetch newsworthy Twitter account data on short notice...

And those aren't even all the useful tools that that team has created...Jeremy Singer-Vine (their data editor) has several great Python utilities in his personal repo (github.com/jsvine), including waybackpack, pdfplumber (a wrapper around pdfminer and similar to Ruby's tabula), and markovify.

[1] https://github.com/BuzzFeedNews/H-2-certification-data

[2] https://github.com/jsvine/twick

There are no comments on this comment yet.