Re:Using Node.js and JQuery to Crawl Public Tweets
by blyxa on Sunday August 05, 2012 @08:42PM
(#4343225)
why not use the twitter api?
[ Parent
| on HN ]
Re:Using Node.js and JQuery to Crawl Public Tweets
by bdreadz on Sunday August 05, 2012 @08:54PM
(#4343252)
from the github page:
Birdeater does not use Twitter's API. It was built as a demonstration of an approach I like to use for parsing structured information from unstructured HTML.
Re:Using Node.js and JQuery to Crawl Public Tweets
by TazeTSchnitzel on Sunday August 05, 2012 @09:04PM
(#4343274)
A better (and practical) example is scraping an internet forum (I've done it, partially)
Re:Using Node.js and JQuery to Crawl Public Tweets
by wamatt on Sunday August 05, 2012 @10:01PM
(#4343400)
The API probably has a key that can be blocked.
Not that I'm advocating this, but a potential advantage of scraping, is it can be combined with tor or proxies, etc to get around limits.
Re:Using Node.js and JQuery to Crawl Public Tweets
by dmazin on Sunday August 05, 2012 @11:29PM
(#4343582)
It's rate-limited to 150 requests per hour and you can only pull the 3,200 most recent tweets for a user.