News for hackers. Stuff that matters.
Re:Analyzing IMDB Data on 90,000 TV Series by BenderV on Sunday June 15, 2014 @09:19PM (#7896965)
"Using a simple scraper written in Ruby, I was able to grab data on the >90,000 TV shows in the IMDB database"

I know it may be a bit more hard to use than a crawler (sadly), but IMDB give their dataset for free (under conditions) : http://www.imdb.com/interfaces

(imdb stricly forbit crawler)

edit : nice analysis ! thanks !

Re:Analyzing IMDB Data on 90,000 TV Series by yaeger on Monday June 16, 2014 @09:04AM (#7898796)
True, but I think the rational behind that is that imdb doesn't want someone creating a software that gets used by a lot of people which implements such a crawler.

I doubt although I can't say for certain, that imdb much cares about someone who made a scraper to gather data for a one time aggregation like this.

Of course, for a use case like OP has, the static data imdb offers would also have sufficed, I'd say. Of course, that data is not as "up to date" as the data you can scrape off of the web pages, but it would have been sufficiently "up to date" for such an analysis, I think.