Re:DataSift Architecture: Realtime Datamining at 120,000 Twe...
by geuis on Tuesday November 29, 2011 @07:43PM
(#3293089)
I find the architecture to be really interesting and useful to learn from. However, they are way too expensive. 1000 tweets is not very much data. I'm building a realtime app now and easily am processing tens of thousands of relevant tweets a day. While a service like Datasift could alleviate a lot of heavy lifting on my part, the cost just doesn't make up for it. It feels like their business model is currently focused on use-cases requiring highly specific targeting, but not intended for use where services need high volumes of certain types of data. Shame, that.
[ Parent
| on HN ]
Re:DataSift Architecture: Realtime Datamining at 120,000 Twe...
by quipo on Tuesday November 29, 2011 @11:22PM
(#3293707)
@geuis: the cost for 1000 tweets is $0.10, and that's the Twitter license fee. You get $10 in free credits when you sign up, and you can have an On Demand plan starting from $10. It should definitely make it very affordable to anyone wanting to play with the Twitter firehose, the barrier-to-entry has never been so low.
Re:DataSift Architecture: Realtime Datamining at 120,000 Twe...
by nirvana on Tuesday November 29, 2011 @10:02PM
(#3293478)
Where are you getting your tweet feed? My initial interest in data sift was because they let you get a feed from the firehose. Twitter doesn't seem to let you do this.
All I want is tweets relevant to a particular subject, and in the early days I don't want to be paying hundreds of dollars for it... I've got keywords and phrases I can use to find them, if I just had access to an API that would let me. (Maybe twitter offers this, I couldn't find it in the past.)
Re:DataSift Architecture: Realtime Datamining at 120,000 Twe...
by geuis on Tuesday November 29, 2011 @10:43PM
(#3293583)
I'm using the stream, but using track filtering. So far it's working very well for my purposes. You are right in that different use cases might need the firehouse and that's where services like gnip and datasift really come in handy. It's too bad that there's not a middle ground.
Re:DataSift Architecture: Realtime Datamining at 120,000 Twe...
by Titanous on Tuesday November 29, 2011 @10:31PM
(#3293551)
If you are just looking for keyword streams, they're available for free: https://dev.twitter.com/docs/streaming-api/methods