I just updated Tornado's httpclient to include a streaming_callback parameter which allows you to process data as it arrives (the param was somehow lost when httpclient was extracted from the FriendFeed code).
To use it, just do something like:
http_client.fetch("http://stream.twitter.com/1/statuses/filter.json?follow=IDS...,
cb, streaming_callback=self._handle_data, auth_username=...)
def _handle_data(self, data):
lines = (self.prev_linepart + data).split("\n")
self.prev_linepart = lines[-1]
lines = lines[:-1]
if not lines:
return
for t in lines:
try:
tweet = tornado.escape.json_decode(t)
I just updated Tornado's httpclient to include a streaming_callback parameter which allows you to process data as it arrives (the param was somehow lost when httpclient was extracted from the FriendFeed code).
To use it, just do something like: