Re:Reconstructing Twitter's Firehose
by keithwinstein on Thursday February 28, 2019 @03:58AM
(#19270023)
I believe your calculations are too high by a factor of 1000... 53 bytes/(typical tweet) * 6000 typical tweets/second is 318 KB (or 0.3 MB) per second, not 318 MB.
Yikes, you are totally right. Fixed
Since a couple of years ago the tweet limit was bumped to 280 characters, so I assume the amount is double of what you calculated.
This is if you only collect username, timestamp and tweet, excluding any additional metadata such as data center, likes and retweets, not to mention images and videos.
Full calculation here: https://docs.google.com/spreadsheets/d/1BIAguT9Qvy0GK-dalpQf...