News for hackers. Stuff that matters.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by bdb on Tuesday December 14, 2010 @04:14PM (#2005763)
What happens when Amazon launches their competing streaming video service? [1] Just a rumor at this point, but when a large set of content comes in-window for both Netflix and Amazon and each needs to spin up a thousand instances to transcode it? Who will get priority? [2]

It seems extraordinarily stupid to entrust such an obvious potential competitor with the keys to your infrastructure, particularly in a single-vendor solution.

This is fundamentally different from the Roku model -- neither Amazon nor Netflix own Roku, so they can remain neutral.

(edit: formatting)

[1] http://technologizer.com/2010/08/31/amazon-subscription-stre...

[2] And don't give me any bullshit about AWS not being capacity constrained unless you routinely spin up 1000 temporary XL instances.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by wheels on Tuesday December 14, 2010 @04:52PM (#2005873)
Do you seriously think this didn't occur to the Netflix management prior to the switchover?

A couple points:

• Netflix pulling out amid internet drama of anti-competitive behavior would hurt Amazon much worse than it would hurt Netflix. Netflix can find someone to throw their money at to host their infrastructure, but they could deal a pretty hefty blow to Amazon's emerging revenue stream in infrastructure by scaring lots of potential customers away.

• There have been talks about a some-day Amazon / Netflix merger for a bazillion years. This could be a little technical lubricant to such a deal.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by bdb on Tuesday December 14, 2010 @07:14PM (#2006390)
Netflix is merely a peripherally-involved third party in the recent Comcast/Level(3) drama. But I guess being a big enough AWS customer might help them get problems fixed when they arise.

I sincerely doubt that Netflix would be acquired by Amazon without a serious downward adjustment to their market cap. Besides, Amazon hosting the entire Netflix infrastructure would present a dramatic disadvantage to any other potential acquirer; in general, M&A yields the best returns for shareholders of the acquired company when there is a bidding war taking place.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by samratjp on Tuesday December 14, 2010 @04:31PM (#2005801)
"It seems extraordinarily stupid to entrust such an obvious potential competitor with the keys to your infrastructure, particularly in a single-vendor solution."

If this (http://www.google.com/finance?client=ob&q=NASDAQ:NFLX) is any indication, I am sure they are secretly building their farm somewhere as we speak.

On a side note, I wonder why all these companies brag about AWS so much (yeah, yeah, AWS is awesome, I use it myself too) - are they getting endorsement incentives or something?

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by brown9-2 on Tuesday December 14, 2010 @10:53PM (#2006974)
Netflix has recently started a "tech blog" and is keeping it regularly updated with articles, although so far they mostly seem to be about "How Netflix uses tech" rather than talking about their code, OSS projects, low-level design decisions, etc.

I think they're starting to discuss these things a bit more as a recruitment strategy.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by ianhawes on Wednesday December 15, 2010 @12:58AM (#2007223)
Thats how I read the entire blog post...A recruitment effort.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by bdb on Tuesday December 14, 2010 @07:02PM (#2006340)
From what I have heard, they are not. They are literally going full speed ahead moving all of their infrastructure (aside of streaming) over to AWS.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by dkl on Tuesday December 14, 2010 @05:12PM (#2005939)
I am a very casual user of AWS and have been told many times that AWS couldn't start an image of the type I requested in the zone I requested it. That's the problem with using EBS-backed images, you can only start them in specific zones.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by lrm242 on Tuesday December 14, 2010 @06:41PM (#2006249)
It's a trivial matter to snapshot your EBS instance to S3, and start an instance somewhere else using that snapshot.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by parenthesis on Tuesday December 14, 2010 @06:48PM (#2006275)
> What happens when Amazon launches their competing streaming video service?

Amazon already happily takes a small cut from competitors' businesses: from the many third party sellers who sell through Amazon's retail sites.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by bdb on Tuesday December 14, 2010 @07:00PM (#2006325)
Businesses where Amazon is already dominant, or at least well-established, yes. Amazon also consistently promotes their own first-party sales above third-party sales. Many of those third-party sellers also have other outlets for selling their product.

I'm not sure your analogy stands up.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by maukdaddy on Tuesday December 14, 2010 @02:37PM (#2005415)
We could have chosen to build out new data centers, build our own redundancy and failover, data synchronization systems, etc. Or, we could opt to write a check to someone else to do that instead.

If you are building almost any type of company today, it makes no sense to build your own datacenter. We're going to look back in 10-20 years and laugh that every company felt the need to build datacenters. While doing some research relating to sustainability and cloud computing, I found some statistics showing that federal agencies saved between 25-50% by moving to "cloud" solutions.

http://www.brookings.edu/papers/2010/0407_cloud_computing_we...

Cloud computing is an excellent way for companies to dramatically lower their carbon footprint as well. Instead of using coal-fired electricity running your own inefficient datacenters, why not host your stuff at Google or Amazon where the PUE is low and they're using hydro/cleaner power?

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by RyanGWU82 on Tuesday December 14, 2010 @03:30PM (#2005620)
If you are building almost any type of company today, it makes no sense to build your own datacenter.

Except for when it _does_ make sense.

I'm starting a project at my company to add a single server in-house, and move some data storage away from Amazon S3. This will save us thousands of dollars a month -- the payback period for the project is 3 months. (!) That includes the engineering cost to set up the new server. This particular project is compelling because we don't need S3's other advantages, we just need raw storage.

Moral: Don't be dogmatic about decisions in your business. Just weigh the pros and cons (including cost) carefully.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by maukdaddy on Tuesday December 14, 2010 @03:52PM (#2005695)
Hence the reason I said "almost". There will always be exceptions.

When looking at a cost/benefit don't forget the following costs:

  Power
  Cooling
  Space (lease costs)
  Administration
  Backup
  Redundancy
  Decommissioning (you don't just throw the server away do you?)
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by foobarbazetc on Monday December 20, 2010 @09:17PM (#2026629)
Price/performance is absolutely terrible on AWS.

What most people don't realize and/or talk about with AWS is how much more expensive it is than in-house/co-lo/dedicated elsewhere.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by chime on Tuesday December 14, 2010 @03:22PM (#2005583)
> We're going to look back in 10-20 years and laugh that every company felt the need to build datacenters.

Until recently, I don't think it was "felt the need to" as much as "had no choice but to." AWS-style data-center is a new development in technology. The ability to remotely fire up servers and have them running in minutes is wonderful but it also means you have to redo your entire IT architecture. For companies just starting up, it's no big deal. For ones already established for 30 years, it is nothing short of a Herculean effort. Setting up a datacenter might be a better choice.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by maukdaddy on Tuesday December 14, 2010 @03:33PM (#2005628)
Very good point, and a poor choice of words on my part.

OTOH, there are companies now that can handle mainframe SaaS ( outsourcing your mainframe apps). Huge gains for companies that want to take the leap and dismantle their mainframes.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by nradov on Tuesday December 14, 2010 @05:04PM (#2005908)
HIPAA privacy and security rules are still a significant obstacle for hosting US healthcare applications on third-party clouds. Those obstacles will eventually fade away as the covered entities become more comfortable with cloud security, and the cloud providers offer standardized business associate agreements. But it may take 10 years or more until we see widespread adoption.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by cwinters on Tuesday December 14, 2010 @09:40PM (#2006813)
Have you tried to go through this process? Amazon seems to have done a good deal of work to ensure its servers and practices are HIPAA-compliant: http://aws.amazon.com/about-aws/whats-new/2009/04/06/whitepa...

I think, as you allude, that the primary issue is not technical but behavioral. In the area of healthcare I work in (nursing homes), quite a few organizations have their data hosted elsewhere. Sometimes it's through the vendor, sometimes it's with a third-party service. I don't see that big a leap of faith to Amazon from this.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by nradov on Thursday December 16, 2010 @07:10PM (#2014493)
I haven't tried. Our prospective customers often want to physically inspect our data center as part of their due diligence process, and I can't see how Amazon could accommodate that.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by ars on Tuesday December 14, 2010 @08:08PM (#2006556)
I think you mean a higher PUE.

Anyway, a better PUE will certainly help your carbon footprint, but saying "using hydro/cleaner power" is meaningless.

You don't get to pick and choose "I'm using hydro, you're using coal" - the power grid is interconnected - everyone is using everything.

Additionally all the possible hydro that can be used already is, so it's not like you can individually promote hydro usage - if you "use" hydro power, all the happens is that someone else doesn't get to use it, which means effectively you are burning coal, but remotely.

There are some slight benefits to using the power near where it's generated to avoid transmission losses, but it's not large.

Unless you can say "because of me there is a new, cleaner, power station that is built" you are not using cleaner power, you are just shifting less clean power to someone else.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by akg_67 on Tuesday December 14, 2010 @05:14PM (#2005948)
After what Amazon and other US providers did to WikiLeaks, I will recommend everyone to have a backup plan to outsourcing data center to 'cloud'. And, backup doesn't mean another instance on Amazon cloud or another similar cloud provider.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by byteclub on Tuesday December 14, 2010 @02:45PM (#2005455)
I think what that post is trying to say is this: focus on your core competency. Amazon is good at cloud computing, Netflix is good at renting/streaming movies.

However, there are some cases where "core competency" is a vague term. Take Facebook for example - could they outsource their infrastructure to somebody else? Given how much data they have to handle, probably not. When you get to the point where scaling requires very low-level (Hunch) or massive (Google or Facebook) optimizations/deployments, you might be entering the business of handling intricate infrastructure yourself.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by SRSimko on Tuesday December 14, 2010 @03:04PM (#2005525)
I work at an IaaS company and we hear the same thing daily most want to focus on what they do best and often it's not server hugging.

Also companies that embrace the cloud have changed how they do things because they see the efficiencies and recognize that they don't want to be in the business of managing servers.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by tcskeptic on Tuesday December 14, 2010 @05:05PM (#2005911)
I look in on this stuff from the enterprise side of things with interest. I want to get out of the business of hosting an internal datacenter entirely as soon as possible (that decades old, and apparently intractable AS400 accounting system notwithstanding), and would prefer to move things cloudward. Right now though, it seems that we are either limited to niche players or proprietary solutions(AWS, etc.), and the risk of being held hostage by an underperforming vendor remains high. We are experimenting with certain applications (CRM, for example) being hosted in the cloud. Until there is a standard cloud-hosting platform, with multiple large players offering services that would allow us to very simply move from one service to another without redisigning major parts of applications there won't be much movement to the cloud in our space. (part of this is a legacy of horrendous application design in the enterprise space, part of it is just the natural legacy of having much older legacy applications than web companies do, and part of it is a reflection of large organizations being terribly risk averse.)
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by somic on Tuesday December 14, 2010 @03:19PM (#2005575)
This is indeed an interesting post but it doesn't answer the question why they chose Amazon. It focuses on why they chose not to expand within their own datacenter.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by pchristensen on Tuesday December 14, 2010 @03:37PM (#2005647)
Who would be a viable alternative vendor to Amazon at Netflix's scale?
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by sgk284 on Tuesday December 14, 2010 @04:35PM (#2005817)
Rackspace?
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by jeremyjarvis on Tuesday December 14, 2010 @03:28PM (#2005607)
Agreed, I was interested to see which of Amazon's features were particularly of use to them. As opposed to, say, Rackspace Cloud.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by somic on Tuesday December 14, 2010 @05:09PM (#2005930)
This talk by @adrianco may address the points you were looking for - http://www.blip.tv/file/4252897

To sum up, Netflix's strategy is to go with the best and biggest in each category, and AWS is obviously the biggest as of now.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by brown9-2 on Tuesday December 14, 2010 @10:57PM (#2006989)
If anyone is interested slides from that talk are here: http://www.slideshare.net/adrianco/netflix-on-cloud-combined...
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by epoxyhockey on Tuesday December 14, 2010 @03:29PM (#2005614)
I'm still apprehensive about entrusting a single party to handle all of my infrastructure. Technical and business failures occur in even the best of the hosting companies.

Granted, my computing needs are basic compared to the needs of Netflix.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by wmf on Tuesday December 14, 2010 @03:58PM (#2005714)
In-house IT is also a single party.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by epoxyhockey on Tuesday December 14, 2010 @05:13PM (#2005945)
I think the motivation for in-house IT to succeed is greater than a 3rd party with their SLA. But, I am also coming from the viewpoint that geographically dispersed servers through multiple hosting companies reduces the 'single point of failure' scenario.
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by ewams on Tuesday December 14, 2010 @04:07PM (#2005738)
Using one (or a mix) of hardware vendors entrusts your infrastructure to a single party as well. Unless you are making your own hardware?
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by frossie on Tuesday December 14, 2010 @03:43PM (#2005663)
"We’re not very good at predicting customer growth or device engagement."

But they are good enough to admit that, and plan accordingly. Hat off.

Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by shimi on Tuesday December 14, 2010 @03:16PM (#2005561)
Netflix have done the same thing with Roku, they like to focus on what they do best
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by redwood on Tuesday December 14, 2010 @04:32PM (#2005802)
Amazon has plenty of spare cycles most of the year (hence Mechanical Turk and all this excess datacenter capacity they can contract out). - They have this b/c they need to be able to accommodate their extrmeley peaky demand (i.e. holiday shopping season!!!) - I wonder if this means netflix streaming will be significantly for flaky in the run-up to Xmas :)
Re:Why Netflix Chose Amazon’s Cloud as Their Computing Platform by wmf on Tuesday December 14, 2010 @05:02PM (#2005903)
EC2 doesn't run on spare capacity (although perhaps Amazon buys from EC2 in competition with other customers) and Netflix streaming doesn't run on EC2 anyway.