News for hackers. Stuff that matters.

Ask HN: What does the Twitter/Datasift deal mean for developers?

So, I write for ReadWriteWeb and I've been paying pretty close attention lately to the situation with Twitter, API access, the API Terms of Service and the developer ecosystem. Earlier this week, Twitter announced a deal with DataSift (formerly TweetMeme) to sell access to Twitter info (including historical) at a price much lower than the full firehose access offered with Gnip. The access is based on search, comes with extra data from Klout, PeerIndex, Qwerly and Lexalytics.

(Here's our writeup of the announcement too: http://www.readwriteweb.com/archives/twitter_announces_fire_hose_marketplace_up_to_10k.php )

So, I'm looking for reactions and interpretation from developers on what this means for the Twitter developer ecosystem. Got one? Got some insight? Let's talk!

( 10 comments | 9 points | on HN )
Re:Ask HN: What does the Twitter/Datasift deal mean for deve... by rwwmike on Wednesday April 06, 2011 @11:48PM (#2418113)
Oh, and I'm Mike. My email is [email protected] if you don't want to just comment here. If you are willing to be quoted for a story, I'll need to know your full name and position/job title/company, what have you. :)
Re:Ask HN: What does the Twitter/Datasift deal mean for deve... by doug_williams on Thursday April 07, 2011 @07:11PM (#2421620)
Mike -- Your statement is factually false. Both DataSift and Gnip have two components of the cost they charge consumers for their search-based products. The first is a data license cost (10 cents per thousand tweets sent to the consumer) and a processing cost. In the case of Gnip, the processing cost is essentially the cost of the machine you license, while for DataSift it is the AWS-style compute cost. How they show these costs to the consumer may vary, but they are each compensate Twitter for the cost of the data license.

To simplify:

<total cost> = <data cost> + <processing cost>

<total cost> = <$0.10 * #tweets % 1000> + <processing cost levied by Gnip or DataSift>

Thanks, Doug

Re:Ask HN: What does the Twitter/Datasift deal mean for deve... by rwwmike on Thursday April 07, 2011 @02:29AM (#2418455)
Re:Ask HN: What does the Twitter/Datasift deal mean for deve... by jvaleski on Thursday April 07, 2011 @06:59PM (#2421576)
check your facts by reaching out to Twitter directly.

I'm Jud Valeski, CEO of Gnip ([email protected]). So we're all running on factual data, apparently a problem these days,...

Twitter's value added resellers (e.g. Gnip and DataSift) must... - sell filtered data at no less than $0.10/1k Tweets consumed. Gnip makes this price explicit, whereas DS, apparently (I can't fully tell so check w/ them directly) rolls that number into their costs, making it opaque.

- sell only to firms who will not display the Tweets in "public".

- sell only to firms who will not programmatically re-syndicate the raw Tweets further downstream.

Re:Ask HN: What does the Twitter/Datasift deal mean for deve... by rwwmike on Thursday April 07, 2011 @07:40PM (#2421706)
Thanks Jud. I'll email you now. :)
Re:Ask HN: What does the Twitter/Datasift deal mean for deve... by jrhchicago on Thursday April 07, 2011 @12:05AM (#2418139)
My understanding is the primary difference is that data from Datasift is not for display (analysis only), which, if accurate, would only work for a small set of companies currently accessing the data.
Re:Ask HN: What does the Twitter/Datasift deal mean for deve... by jrhchicago on Thursday April 07, 2011 @03:00AM (#2418525)
I'm basing that on this excerpt from the announcement on the Datasift website

"This means that developers interested in building tools to monitor and analyze Twitter data can now filter Tweets from the full Twitter Firehose, using DataSift for non-display analysis."

Re:Ask HN: What does the Twitter/Datasift deal mean for deve... by sinak on Thursday April 07, 2011 @01:20AM (#2418298)
I'm not sure if that's true. Many of their use case examples seem to display the data from user's streams, meaning that it's not just analysis.
Re:Ask HN: What does the Twitter/Datasift deal mean for deve... by doug_williams on Thursday April 07, 2011 @07:14PM (#2421635)
Hi -- The license the consumer receives is strictly for non-display use cases, meaning you are not able to go to Gnip or DataSift to license Twitter content to display to end users. We (Twitter) still maintain a robust set of APIs for display use cases which we will license directly to partners for display products.

Thanks, Doug

Re:Ask HN: What does the Twitter/Datasift deal mean for deve... by rwwmike on Thursday April 07, 2011 @12:25AM (#2418181)
I thought that I had asked them about that part. Hmmm.