This is Derek, from the video. Sorry I am just joing the conversation mow.

For this use case, we setup 800 Gbps of interconnect with Google.

So at 300PB it took about a month to transfer? Sounds like an interesting project in itself.

Yeah! We’ll be operating in a hybrid model steady-state, so the links aren’t just for the one-time copy.

That's insane.

With the interconnect being faster than the network cards by several orders of magnitude (I assume unless there is special hardware), how is it structured to utilize all of the 800Gbps?

We are using dedicated clusters to push data over the links.

A member of our team proposed a session for Google NEXT '19 that goes into the architecture we are using in a lot more depth. We're waiting to see if that talk is accepted or not, but in any case we're planning to share more soon.

Wow!

Migrations like these take time and happen step by step.

First you can try to replicate some data and move read instances. After that step by step the rest.

Its really not possible to do that overnight or months even.

Fyi: im currently migrating twice the Twitter size one in the world infrastructure from aws to azure now.

Isn‘t exporting 600TB of data from aws ridicously expensive?

Twitters number is in PB, not TB. At that scale, assume that none of the normal pricing and procedures apply.

600TB would come out to around $50000 which for Twitter is peanuts. This is their bandwidth pricing though but I guess they'd use snowmobile export (if that exists?) instead.

For reference, 300PB (the amount referenced in the article) using the same pricing would cost $25M. Still might be a reasonable one-time cost for migration for a large corporation, though, even disregarding the "bulk" rates they'd see at that scale.

AWS Snowmobile cost is $0.005/GB per month which comes out to $1.5M/month for 300PB for however long the data is on the truck.

> Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.

https://what-if.xkcd.com/31/

The mention of "SneakerNet" reminded me of old times when we had a bunch of old Macintoshes (pre-OSX) connected with a proprietary local network called LocalTalk. It was so unreliable that we had a couple of 3.5" floppy disks that we used to transfer files when it didn't work; the disks were labelled "LocalWalk". :D