News for hackers. Stuff that matters.
Re:Why Dropbox decided to build its own infrastructure and n... by swampthinker on Friday September 15, 2017 @09:05PM (#15262152)
I'll be 100% honest, I didn't realize Dropbox was still on AWS. You figure at a certain scale it makes more sense to run your own solution.
Re:Why Dropbox decided to build its own infrastructure and n... by praneshp on Friday September 15, 2017 @09:48PM (#15262331)
(Dropbox employee)

I don't think this article is news. The editorialized title in HN is unfortunate and I wish a mod can change it. The article talks about the reasons Dropbox decided to move 500PB storage into our own data centers. Also like the article mentions, "They still use AWS for some workloads".

Edit: Here is another HN submission of the same piece with the correct headline. https://news.ycombinator.com/item?id=15257331

Re:Why Dropbox decided to build its own infrastructure and n... by xbmcuser on Friday September 15, 2017 @10:07PM (#15262400)
Yeah and that's why it amazes me that Netflix uses Amazon a straight competitor that uses the money it makes from Netflix to compete with Amazon prime.
Re:Why Dropbox decided to build its own infrastructure and n... by StreamBright on Saturday September 16, 2017 @04:30AM (#15263600)
Netflix probably in a better position to do the math than random people on HN. I think they have done their homework and decided that it is overall cheaper to keep ___some___ of their infra on AWS.
Re:Why Dropbox decided to build its own infrastructure and n... by dna_polymerase on Saturday September 16, 2017 @10:41AM (#15264825)
Yes! Thank you! Every time some piece like this is posted 'experts' arise here and know everything better. Like there is a bunch of idiots working at Netflix who are happily burning money...
Re:Why Dropbox decided to build its own infrastructure and n... by late2part on Sunday September 17, 2017 @12:24AM (#15268229)
Re:Why Dropbox decided to build its own infrastructure and n... by kabes on Saturday September 16, 2017 @02:35AM (#15263265)
But this happens all the time. Like Apple buying iPhone components from Samsung. The best deal is the best deal
Re:Why Dropbox decided to build its own infrastructure and n... by mulmen on Friday September 15, 2017 @10:14PM (#15262428)
I hear this line of thinking a lot but I don't really buy it.

You could just as easily say it's amazing that Amazon allows a competitor like Netflix to run on it's platform. The reality is that relationship is more complex than that.

Re:Why Dropbox decided to build its own infrastructure and n... by kinkrtyavimoodh on Friday September 15, 2017 @11:52PM (#15262759)
Netflix is going to exist regardless of whether Amazon lets them run on AWS.

Amazon letting them run on AWS is brilliant as they get a piece of Netflix's pie. So even if Netflix beats Amazon Prime, Amazon gets to dip into their pot via revenues from AWS. It's a great hedging of bets.

Re:Why Dropbox decided to build its own infrastructure and n... by peanutgallery27 on Saturday September 16, 2017 @04:52AM (#15263672)
They're totally separate businesses. The people that treat Netflix like their customer aren't the same people making strategic decisions about Amazon Prime. Amazon is a ridiculously large company.
Re:Why Dropbox decided to build its own infrastructure and n... by siculars on Saturday September 16, 2017 @09:37AM (#15264603)
They are not totally separate when the P&L roll up on the same spreadsheet.
Re:Why Dropbox decided to build its own infrastructure and n... by user5994461 on Saturday September 16, 2017 @02:41PM (#15265975)
That depends on how the Amazon business is organized. It's possible that the P&L never join on the same spreadsheet.
Re:Why Dropbox decided to build its own infrastructure and n... by ryacko on Saturday September 16, 2017 @02:24AM (#15263229)
The really amazing thing is that Nestle licensing production of Kit Kats to Hershey.
Re:Why Dropbox decided to build its own infrastructure and n... by dec0dedab0de on Saturday September 16, 2017 @08:01AM (#15264230)
I think that deal was done before nestle bought kitkat. Kind of like spiderman movies being with sony instead of disney
Re:Why Dropbox decided to build its own infrastructure and n... by user5994461 on Saturday September 16, 2017 @02:39PM (#15265969)
Migrations and self hosted infrastructure don't do themselves. Netflix can get quite a lot of stuff done on AWS for the $200k+ dollars it would take to hire one single employee.
Re:Why Dropbox decided to build its own infrastructure and n... by amorphid on Saturday September 16, 2017 @01:39AM (#15263096)
What's the largest cloud provider that does NOT offer a Netflix-like streaming video service? Amazon/AWS, Microsoft/Azure, and Google Cloud all have offer premium streaming content of some type. IBM Cloud?
Re:Why Dropbox decided to build its own infrastructure and n... by dingo_bat on Saturday September 16, 2017 @04:25AM (#15263583)
Microsoft has a Netflix-like service?
Re:Why Dropbox decided to build its own infrastructure and n... by fragmede on Saturday September 16, 2017 @07:26AM (#15264117)
I presume amorphid is referring to the tech stack supported in Azure and not a consumer facing service.

https://azure.microsoft.com/en-us/services/media-services/

(MSNBC would be a video streaming service that bears Microsoft's initials, but they sold off their stake in 2012)

Re:Why Dropbox decided to build its own infrastructure and n... by amorphid on Saturday September 16, 2017 @08:45AM (#15264401)
They have premium content which can be streamed. Perhaps not Netflix-like in there business model.

https://www.microsoft.com/en-us/store/movies-and-tv

Re:Why Dropbox decided to build its own infrastructure and n... by xbmcuser on Saturday September 16, 2017 @03:15AM (#15263371)
My point was that Netflix uses AWS services exclusively instead of dividing it between multiple services at least though now days they probably get really deep discounts from Amazon to stay on AWS because of competition and it probably is lot cheaper for them then building and maintaining it's their own servers.
Re:Why Dropbox decided to build its own infrastructure and n... by lstyls on Saturday September 16, 2017 @07:43AM (#15264171)
Netflix built their own CDN ages ago. Given that their core business logic is basically static content delivery whats left to AWS probably is a lot less than commenters in this thread seem to think.
Re:Why Dropbox decided to build its own infrastructure and n... by notyourday on Saturday September 16, 2017 @10:53AM (#15264880)
CAF
Re:Why Dropbox decided to build its own infrastructure and n... by dandr01d on Friday September 15, 2017 @09:13PM (#15262189)
This article isn't an announcement - it's a "Why we did it". They started transitioning years ago.
Re:Why Dropbox decided to build its own infrastructure and n... by rizjoj on Friday September 15, 2017 @09:11PM (#15262176)
and Netflix?
Re:Why Dropbox decided to build its own infrastructure and n... by kyledrake on Friday September 15, 2017 @09:15PM (#15262200)
They run BGP and do an enormous amount of IX peering. https://www.peeringdb.com/asn/2906

AWS doesn't even let you do BGP. If you want to use them for a CDN, you're monopolized into their network and their blend of ridiculously overpriced bandwidth.

They might be using AWS for the canonical store of data, but the sanity of using "the cloud" goes out the window the second you need to ship a lot of traffic to the public internet.

Re:Why Dropbox decided to build its own infrastructure and n... by jimktrains2 on Friday September 15, 2017 @09:37PM (#15262295)
> heir blend of ridiculously overpriced bandwidth.

For someone not at netflix's level, can you recommend somewhere with significantly cheaper bandwidth.

Re:Why Dropbox decided to build its own infrastructure and n... by NightlyDev on Friday September 15, 2017 @11:03PM (#15262609)
While I don't know what pricing AWS has for high bandwidth customers since it's not public, I do know that the public prices are really high.

The thing is, even for smaller companies it's easy to get way better deals on bandwidth. I'm currently using a provider that runs openstack and I'm paying less than $0.009/GB, from the first byte(no commitment). I'd say that is very cheap compared to AWS pricing at $0.05/GB above 350 TB.

With colocation or renting dedicated servers and IP-transit you can get way lower prices. A dedicated 1 Gbps is about 300 USD/month with my provider, and the prices drops hard for dedicated 10, 40 and 100 Gbps uplinks.

Never used OVH myself, but they are providing 2 Gbps at £340/month for their highest quality bandwidth, and £64/month for bandwidth they think would be good for downloads.

I think using cloud providers for somewhat high bandwidth services are basically throwing money out the window. Until prices really drops(if they ever will) I'll continue thinking that clouds are for temporary workloads or prototyping with the ability to easily scale at a high cost.

Re:Why Dropbox decided to build its own infrastructure and n... by CharlesW on Saturday September 16, 2017 @12:36PM (#15265358)
> I'm currently using a provider that runs openstack and I'm paying less than $0.009/GB, from the first byte(no commitment).

Would you mind sharing who that is?

Re:Why Dropbox decided to build its own infrastructure and n... by NightlyDev on Monday September 18, 2017 @01:58AM (#15273669)
I'm not allowed to say, but I think OVH also runs openstack with the "public cloud" offering, and I think bandwidth are even cheaper with them.
Re:Why Dropbox decided to build its own infrastructure and n... by user5994461 on Saturday September 16, 2017 @02:43PM (#15265989)
It's interesting that what you consider a great provider is only doing 6 times cheaper than AWS. That's not much of a discount.
Re:Why Dropbox decided to build its own infrastructure and n... by notyourday on Saturday September 16, 2017 @11:05AM (#15264935)
Below $10k/mo bandwidth plays are irrelevant - the cost of the support infrastructure is just too high.

At $10k/mo what you do is plop a rack in one of the well connected buildings in NYC, VA, SF, SJC, CHI and get either flat rate 10Gs or 1G commits over 10Gs. You should be paying between 0.55 and 0.75 per Mbit/sec on 95th percentile on 1G commit. You put into the same cabinet your edge nodes that actually would be pushing the traffic out and use AWS or GCP for your compute workloads.

Re:Why Dropbox decided to build its own infrastructure and n... by corford on Friday September 15, 2017 @10:25PM (#15262467)
Pretty much anyone... Hetzner, Linode, Choopa
Re:Why Dropbox decided to build its own infrastructure and n... by jimktrains2 on Friday September 15, 2017 @10:33PM (#15262504)
For those I need to run a VPS, which also costs money and has fewer PoP and more mangment overhead.

EDIT: I guess I was talking about CloudFront specifically, not EC2. EC2 is pretty pricey.

Re:Why Dropbox decided to build its own infrastructure and n... by corford on Saturday September 16, 2017 @06:52AM (#15264035)
Sorry misunderstood. You could try here https://www.cdnperf.com/ and compare egress prices/limits.
Re:Why Dropbox decided to build its own infrastructure and n... by dasil003 on Saturday September 16, 2017 @01:28AM (#15263064)
I built a global video CDN using SoftLayer a couple years ago and ended up paying around $0.01/GB.
Re:Why Dropbox decided to build its own infrastructure and n... by jimktrains2 on Saturday September 16, 2017 @11:06AM (#15264947)
Re:Why Dropbox decided to build its own infrastructure and n... by dasil003 on Saturday September 16, 2017 @04:27PM (#15266574)
At the time they included 20TB of traffic per month per server, and it went into regional pools. I think they discontinued that because of us :)
Re:Why Dropbox decided to build its own infrastructure and n... by toomuchtodo on Friday September 15, 2017 @10:16PM (#15262431)
Any colo or dedicated server provider.
Re:Why Dropbox decided to build its own infrastructure and n... by jimktrains2 on Friday September 15, 2017 @10:32PM (#15262496)
At a CoLo I have the rack, managment, and hardware overhead too, though? Also fewer PoP.

EDIT: I guess I was talking about CloudFront specifically, not EC2. EC2 is pretty pricey.

Re:Why Dropbox decided to build its own infrastructure and n... by toomuchtodo on Friday September 15, 2017 @10:35PM (#15262510)
Price compare various CDNs. It's a commodity business. Fastly comes to mind, but we had quite a few outages with them at my last org.
Re:Why Dropbox decided to build its own infrastructure and n... by thezilch on Friday September 15, 2017 @10:18PM (#15262441)
Pretty much anywhere.
Re:Why Dropbox decided to build its own infrastructure and n... by shaklee3 on Saturday September 16, 2017 @01:51AM (#15263131)
Packet.net
Re:Why Dropbox decided to build its own infrastructure and n... by _pmf_ on Saturday September 16, 2017 @11:25AM (#15265033)
> you're monopolized into their network and their blend of ridiculously overpriced bandwidth.

Wouldn't a customer the size of Netflix be granted more liberty in this regard?

Re:Why Dropbox decided to build its own infrastructure and n... by pravinva on Friday September 15, 2017 @09:32PM (#15262275)
What are you on about? AWS practically requires BGP. Static routing is rarely the best choice when you set up DX.

Netflix rolls its own CDNs because Cloudfront can't meet all their demands, yet.

Re:Why Dropbox decided to build its own infrastructure and n... by andyfleming on Friday September 15, 2017 @09:16PM (#15262205)
To be fair, Netflix only runs some of their infrastructure on AWS. Basically, if I understand correctly, their applications run on AWS and their delivery is through their own CDN built on top of various ISPs[1].

[1] https://media.netflix.com/en/company-blog/how-netflix-works-...

Re:Why Dropbox decided to build its own infrastructure and n... by yeukhon on Friday September 15, 2017 @10:50PM (#15262561)
Correct, but some is probably an understatment. They are pretty much run everyting on AWS except what you cited (e.g video processing and content delivery). Netflix is the biggest AWS user.
Re:Why Dropbox decided to build its own infrastructure and n... by scurvy on Saturday September 16, 2017 @12:56AM (#15262961)
Everything on AWS except their largest, most critical piece of infrastructure that needs to be the highest performing. That says it all right there.
Re:Why Dropbox decided to build its own infrastructure and n... by erikbye on Saturday September 16, 2017 @05:00AM (#15263698)
https://www.networkworld.com/article/3037428/cloud-computing...

And,

> We rely on the cloud for all of our scalable computing and storage needs — our business logic, distributed databases and big data processing/analytics, recommendations, transcoding, and hundreds of other functions that make up the Netflix application. Video is delivered through Netflix Open Connect, our content delivery network that is distributed globally to efficiently deliver our bits to members’ devices.

Source: https://media.netflix.com/en/company-blog/completing-the-net...

Netflix still operates its own data centers and CDN to deliver video. This is not done on AWS.

Re:Why Dropbox decided to build its own infrastructure and n... by yeukhon on Saturday September 16, 2017 @09:38AM (#15264604)
Isn't that what I said above?
Re:Why Dropbox decided to build its own infrastructure and n... by eikenberry on Friday September 15, 2017 @09:21PM (#15262227)
They have a completely different use case that benefits from being able to scale up and down quickly. I don't see them moving away from AWS anytime soon.
Re:Why Dropbox decided to build its own infrastructure and n... by bsenftner on Saturday September 16, 2017 @04:08AM (#15263531)
Please, how fucking stupid. Of course they moved as much as possible off AWS, and only use AWS for new and experimental services. The markup of cloud servers is immoral, and the brainwashing that cloud services are a value at all is a testimony to how many people simply do not do the math.
Re:Why Dropbox decided to build its own infrastructure and n... by StreamBright on Saturday September 16, 2017 @04:29AM (#15263596)
I am not sure what you are talking about. They moved out the 500PB data storage because ______at that scale______ it is cheaper to run it on your own hardware. This implies few things like having engineers with skills that are really not easy to find for starting and a bunch of other challenges that you need to solve.