Twitter migrates data to Google Cloud

Re:Twitter migrates data to Google Cloud by chatmasta on Wednesday January 23, 2019 @11:31AM (#18979688)

So at 300PB it took about a month to transfer? Sounds like an interesting project in itself.

[ Parent | on HN ]

Re:Twitter migrates data to Google Cloud by derekalyon on Wednesday January 23, 2019 @12:07PM (#18980080)

Yeah! We’ll be operating in a hybrid model steady-state, so the links aren’t just for the one-time copy.

[ Parent | on HN ]

Re:Twitter migrates data to Google Cloud by AndrewKemendo on Wednesday January 23, 2019 @01:31PM (#18981072)

That's insane.

With the interconnect being faster than the network cards by several orders of magnitude (I assume unless there is special hardware), how is it structured to utilize all of the 800Gbps?

[ Parent | on HN ]

Re:Twitter migrates data to Google Cloud by derekalyon on Wednesday January 23, 2019 @01:56PM (#18981383)

We are using dedicated clusters to push data over the links.

A member of our team proposed a session for Google NEXT '19 that goes into the architecture we are using in a lot more depth. We're waiting to see if that talk is accepted or not, but in any case we're planning to share more soon.

[ Parent | on HN ]