Re:Dragonfly: Alibaba P2P file distribution system
by justinsaccount on Wednesday June 13, 2018 @11:45AM
(#17304147)
It's a cool project, but I'm not sure how huge that is.
If you have 5000 nodes running this, that could be as small as distributing 23.8 gigs/day to each node over the course of the month.
Re:Dragonfly: Alibaba P2P file distribution system
by oh_sigh on Wednesday June 13, 2018 @01:24PM
(#17305184)
3.4PB/month is only 111,000 GB/day. That's not particularly huge in this day and age, except to consumers.
Re:Dragonfly: Alibaba P2P file distribution system
by Xorlev on Thursday June 14, 2018 @12:13AM
(#17309229)
I imagine this traffic is fairly bursty as well.
Also keep in mind this is a data distribution system. In the case of large data pushes, new builds, etc. it's important that all peers get the new data on a timely and reliable manner.
I suspect you're underestimating the problem this project solves by focusing on a mostly irrelevant data rate stat.
Re:Dragonfly: Alibaba P2P file distribution system
by justicezyx on Wednesday June 13, 2018 @12:37PM
(#17304664)
> The reliability is up to 99.9999%
"up to"... Certainly looks like a wrong wording.
Re:Dragonfly: Alibaba P2P file distribution system
by zild3d on Wednesday June 13, 2018 @01:09PM
(#17305020)
I read it in the "we have now gotten it up to X" sense, such as "I'm now up to my 5th beer"
Re:Dragonfly: Alibaba P2P file distribution system
by gcbw2 on Wednesday June 13, 2018 @11:23PM
(#17309014)
except reliability reduces a total, not sum integers like beer count.
if I write a shell script with a scp line right now I can tell you it has 100% reliability. up to X% reliability means that "in the very nice and controlled environment, with the own devs attacking every production problem, trying to get 100% got us up to X%"
Re:Dragonfly: Alibaba P2P file distribution system
by quixoticelixer- on Thursday June 14, 2018 @07:32AM
(#17310849)
Except that is clearly not what they where trying to communicate.
Re:Dragonfly: Alibaba P2P file distribution system
by unicornporn on Wednesday June 13, 2018 @02:58PM
(#17305995)
That's the way to read it.
>At Alibaba, the system transfers 2 billion times and distributes 3.4PB of data every month, it has become one of the most important piece of infrastructure at Alibaba. The reliability is up to 99.9999%.
I took at look at their repo and it turns out there are surprisingly lots of good stuffs in it which never gets much spotlight or attentions.