Re:Twitter migrates data to Google Cloud
by thisgoodlife on Wednesday January 23, 2019 @05:14AM
(#18977073)
How are they gonna move all that 300 pb of data to Google's data center? A fleet of semitrailer trucks?
[ Parent
| on HN ]
Re:Twitter migrates data to Google Cloud
by aboutruby on Wednesday January 23, 2019 @05:21AM
(#18977106)
For instance AWS has 50 Terabytes devices that they ship from your own data center to their data center: https://aws.amazon.com/snowball/
Re:Twitter migrates data to Google Cloud
by kondro on Wednesday January 23, 2019 @05:23AM
(#18977114)
They also have the 100PB Snowmobile.
Re:Twitter migrates data to Google Cloud
by boulos on Wednesday January 23, 2019 @11:56AM
(#18979954)
Disclosure: I work on Google Cloud (with Twitter).
Pipes! That data isn't static either. As Derek said in a sibling comment, they setup hundreds of Gbps of peering. Each 100 Gbps gets you a Petabyte per day (roughly). Planning the move and doing the tooling to keep stuff synchronized takes longer than the actual transfer once you've got say 800 Gbps (just 30 days ish). And again, you're going to have lots of data churn per month, so you want pipes anyway.