Have you actually done this though ? Because I have extensive experience with both Docker and OpenStack and I am sorry but neither comes close to AWS for stability and reliability. They are both really buggy.
Also you seem to be completely missing the either side of AWS which is the software. Using ELB, SNS, SQS, RDS etc. can save a lot of money.
All of my instances in us-east-1 lost outbound connectivity Sunday morning for half an hour, and an hour last night. Tell me about this stellar stability and reliability.
As long as you design your application properly, it'll be reliable wether you run it in AWS, on OpenStack, or even bare metal. StackOverflow does it, you can do it too (they colo all of their equipment in NYC). I have not done this with Docker or OpenStack, but I have using Xen and KVM virtualization (Docker would be about the same as using lxc).
ELB? Haproxy. RDS? MySQL/Postgresql/MS SQL (in that order), unless you absolutely have zero management experience with it. I concede SQS and SNS have no open source counterpart; you'd have to roll your own.
This idea that you can design your application properly and it will be reliable anywhere is ridiculous. If the underlying platform is fundamentally flawed (i.e. more than just nodes coming/going) then your options are very limited. Docker and OpenStack are examples of platforms that are really buggy when used in Production settings.
AWS as a platform is very stable and of a generally high quality. And the components like ELB, RDS etc are managed. Trying to manage the hardware AND all the core infrastructure components on top is more than just one person's job like you mentioned before.
Thats not true at all. If you've designed your application that any instance could fail at any time, it doesn't matter where your instances are hosted: AWS, Digital Ocean, OpenStack, or your own bare metal servers. As long as you've architected the environment to pass traffic to only health nodes (you're doing service checks, aren't you?), healthy microservices (again, service checks), you can promote slaves to masters for your datastores, and you have sufficient capacity for failures, yes, you can design your application to be fault redundant on any infrastructure.
TL;DR If your app is designed for high availability and is already structured to scale out horizontally, its doesn't matter what the underlying infrastructure is.
also how did you managed file access in a scaling manner? did you've setup hdfs? and when yes how did you provided your developers that space? did you created some kind of infrastructure for that (libraries, etc)? There are so many things, which just doesn't work out. As already said, you've just managed it. But other people did the work, that this could work out and they needed a lot of time to get things done. Take a look at google, they always try to bring costs down, through things like borg, kubernetes, etc. but they throw thousands of people at those problems, also at the networking thing, they have a huge team of PROGRAMMERS that try everything that the management of their network will be easier. also if you don't need things all the time, why should you buy it?
> You could manage a thousand servers, but ONE PERSON just can't operate thousands of servers.
Yes, I literally managed AND operated ~5500 linux servers in a facility. Remote power management, serial console access, PXE booting. I would replace any failed hardware when convenient, usually after my morning coffee or in the evening if I wasn't in a rush to get home.
> also how did you managed file access in a scaling manner?
Shared NFS with work distributed to servers on-demand or scheduled, based on workload.
> and when yes how did you provided your developers that space?
Limited SSH access to their environment; diskspace was provisioned in minutes. Scheduled jobs could be executed using an open source scheduling framework.
> did you created some kind of infrastructure for that (libraries, etc)?
No, the majority of the tooling was a few thousand lines of bash scripting
> There are so many things, which just doesn't work out. As already said, you've just managed it. But other people did the work, that this could work out and they needed a lot of time to get things done. Take a look at google, they always try to bring costs down, through things like borg, kubernetes, etc. but they throw thousands of people at those problems, also at the networking thing, they have a huge team of PROGRAMMERS that try everything that the management of their network will be easier. also if you don't need things all the time, why should you buy it?
Not knowing how to do something does not make it impossible or hard, simply an unknown how to do it.
PS: I could see a team of 10 people handling 55,000 servers though. But that's serious scale and fairly lean.
To directly answer your question, people from other teams were available if for some reason I was not.
Facebook is doing 1 engineer to 20K servers: http://www.datacenterknowledge.com/archives/2013/11/20/faceb...
The number of companies who can do that are single digits, but 1 person per ~5000 servers is absolutely reasonable.
I would also be suspicious of 1:10,000+ numbers as they often don't include people that are part of the server team but not standing in the data center. Aka Facebooks 3 person dev team, a manager, whoever designs there HW, the team spun up to do mass HW installations etc.
In the end AWS covers a lot of seperate costs which are only really obvious as you keep scaling up.
I'd like to see a similar analysis for say 50-100 servers, and see if AWS really costs more vs. colocation when you add in the resources to setup and maintain the infrastructure.
I've managed thousands of instances in AWS, and I've managed thousands of servers in colos, both owned (DOE facility, web hosting startup, consulting firm) or with the space leased (Equinix, Savvis, Cogent, IBM).
You could most definitely run elastic workloads in your own datacenter now using Docker containers, and one of the many orchestration frameworks that exist for it.
As I mentioned above, if you don't have a predictable workload, AWS is great. Exceptional even. But if you know your workload and can hire 2-3 people to manage it (which you should be doing at scale), its cheaper to get your own gear and colo it.