The startup I'm working for has minimal scaling required but we still use AWS despite the higher cost for EC2 because the broad ecosystem of AWS products make it easier to develop interesting things quickly and efficiently.

If we went with all of our own dedicated hardware, or cheaper instances from a different cloud provider then we'd miss out on ELB, have slower and more expensive communication to and from S3, not to mention that services like Elastic Beanstalk make deploying to EC2 instances very easy compared with rolling your own deployment system. And for those who don't want to bother with administrating databases and cache machines RDS and Elasticache are going to be cheapest and fastest if your instances are EC2.

So yeah I agree that EC2 is expensive, but the benefits of living fully within the Amazon ecosystem are pretty large.

I think of AWS as a tool for prototyping and early-stage outsourcing of your infrastructure. Use it when you're fighting past market and technology risk with a single-digit team; consider dropping it in order to optimize costs when you've got more people (including fractional people) to evaluate, configure, and operate alternatives.

Isn't relying so much on AWS like "putting all your eggs in one basket"?

AWS != EC2. Do not assume that AWS is only used as VMs. It's different for everyone but AWS provides massive savings for many companies. It makes sense for many use cases in addition to elastic workloads.

True, but outside of S3 and Route53, how much under the AWS umbrella is much use without using at least one EC2 instance?

I can see a lot of benefit to using S3 without EC2, but after that, I'm not sure what else would be possible. Care to elaborate more?

Can you use their queues and database tools w/o using EC2? (If you are using a VPC, maybe?)

Not sure why it has to be without EC2 to be useful. The main benefit is indeed the fact that you can have VMs, queue, load balancers, databases, DNS, cache, etc. all using consistent APIs, etc. You can create dev/test environments that are as close to prod as possible with fraction of the cost, etc.

In our case (and I'm pretty sure we're not in the fringe) when you consider all costs including the administrative overhead, etc. EC2 costs are not significant portion of the cost. Just to give an idea, for us, even if someone had offered VM hosting for free, it would not be cost effective for us to move. For a company that have very high processing requirements it could be a different story. I just wanted to mention that the value added by services in addition to EC2 is sufficiently high that even if EC2 costs are higher than alternative (and they are higher), AWS can still provide significant savings.

Time of a highly skilled dev/ops person is (very) valuable, and not something we can buy more of easily. Anything that saves us time, implement faster pays for itself pretty easily. If you don't have a massive EC2 bill, chances are AWS overall is a good proposition.

Considering just how much more expensive AWS is, I have my doubts on this. AWS also really likes "hiding" costs, like bandwidth costs. It's not obvious that most sites will go over the bandwidth limit and pay extra, whereas for sites on VPS providers, this won't happen. In reality they have a billion different line items on every bill, and most are not mentioned on their pricing pages.

Expect to pay a LOT more than you'd expect to pay from the pricing page.

I don't get the whole line of reasoning : either your project is not important and having weekly or monthly backups is certainly sufficient. In such a case, having a script to backup a VPS is by far the most cost efficient way to run the project.

If your project is important enough to have a complex AWS setup, you have an admin anyway.

"But you can trust amazon". Well, no : http://www.businessinsider.com.au/amazon-lost-data-2011-4 (note that disasters on VPS/dedicated servers are also rare)

If you're making complex AWS setups because "it's cool", then by all means, but keep in mind that you're paying a lot for the privilege. Don't do this on production.

I can only give you a datapoint. This has not been the case in our case. We never paid more than what we expected.

If you think VPS providers is an alternative to AWS, you have an entirely different use case than ours. For an application with high availability requirements, messages queues, scalable data store, etc. admin work is not "monthly backups". Implementing and maintaining highly available load balancing, message queues, data stores, DNS, auto scaling, etc. all require a lot of work, and often skills that you may not have in the team. AWS makes these tasks easier. You still have to do admin work, but much less of it, and don't have to be expert at each one. That's worth a lot.

You're moving the goalposts. First you said:

AWS really only makes sense ($$) when you can take advantage of the ability to spin up and spin down your instances as needed.

After it's pointed out that other facilities besides EC2 motivate people to use AWS, you can't turn around and complain that those other facilities are useless without EC2. Even if that were true (it's not), such people would be using EC2 not for its load-scaling, but rather to enable their use of the non-EC2 facilities.

Who's moving goal posts? After it was (correctly) pointed out that AWS!=EC2 (even though most of the original post was talking about EC2), I asked a simple question - how useful are the rest of the services outside of EC2? It is a legitimate question.

I'm not sure how useful everything in the AWS umbrella is outside of EC2. I was hoping to learn something, not provoke anyone.

I guess I misunderstood you; no worries!

"Can you use their queues and database tools w/o using EC2?" Yes, you very easily can. RDS is a simple setup, and so is SQS (queuing). The same goes for Dynamo and Redshift.

Are you really going tolerate 10+ ms of latency between a database, and an app server? I have never personally tried it, but it seems like it would be a disaster.

I just tested pinging our public RDS instance (non-VPC) from one of our EC2 instances that is loading data ever 1-3 minutes from S3 objects into that RDS instance from within a VPC. 3-5ms ping times. More than acceptable.

It's not 10ms. Halfway through a staged AWS migration now and the latency has really not been an issue. This is from a London data centre to AWS Ireland.

At most it's been 4 to 6 ms. And that has coped with our heaviest load.

Heh, heck my db and app live in the same data center, and the latency is more than 30-40 MS.

Really? That seems enormously too slow. Even over the internet, I'm within 5-10ms of most things in the bay area.

If you have a dozen slow (100us) switches, three slow (2ms) routers, and 500 kilometers of fibre (2ms), you should still be under 10ms each way. And that's enough gear to cross some countries.

I'm curious how you can do much worse than this in a data center. Maybe some pokey "application firewall" or massive buffer bloat?

The problem here is that you're classifying everything AWS does as "EC2", which then causes confusion when someone else comes along and says "vendor X does everything EC2 does for half the price", and they mean solely the VM-related stuff, not the additional array of services.

That's a problem with the comments for this whole post. AWS means a lot of different things to different people. While I focused on EC2, others focused on RDS or S3 or ELB, etc... it's easy to get off on a tangent.

When everything is covered under the AWS umbrella, it's easy to argue from any side you want.

I still think that it's important to consider whether or not AWS (for all values of AWS) correctly fits your workload. You may not need the ability to spin up servers within minutes, or have multi-datacenter redundancy in a database, or have virtually unlimited storage, or a robust queuing system. A lot of great engineering has gone into all of the AWS products, and for many instances it is probably overkill. And that can cost you a lot if you don't know what you're doing.

AWS makes a lot of sense in a 24/7 environment, particularly when you are a new startup and don't have enough information (or capital!) to make educated server purchases.

They don't have to make any purchasing decisions. They can rent servers from companies like Softlayer and Rackspace (#3 and #4 behind AWS for YC startups), or spin up much cheaper VPS's (Linode's #2). We're talking $120/month commitments, not buying hardware and driving to a data center to install it. Deploying to a freshly imaged physical server is the same as deploying to EC2, and they can be provisioned for you in an hour or two. Each of those servers gets you many times the performance of an EC2 instance in the same price class, which means much more time to figure out your capacity needs as you grow.

As someone who has worked w/ AWS (and Rackspace) for several years with multiple startups...

Unless they have dramatically improved their offering in the last couple years, an hour or two from "I need a new server" to delivery is 1) not an accurate timeframe for physical servers from Rackspace and 2) even if it was realistic, that's an eternity when you are trying to iterate quickly. I can have a new server in 30 seconds with AWS and in the course of an hour could have tested my automation tools half a dozen times or more vs reimaging a server over and over again.

I'm not saying it's always the right choice or that it's cheaper, but that flexibility combined with some of the pre-canned tools (ELB, RDS, CloudWatch, SQS, SNS) has tremendous value even when you aren't autoscaling.

You've completely moved the goalposts; "testing automation tools" where your servers live for a few minutes then get destroyed is not the same as "a 24/7 environment". I didn't suggest renting servers to do ephemeral testing.

I only have experience renting physical hardware from Softlayer. They have built and imaged new servers for me in under 2 hours a dozen times, day and night. They also have a "xpress servers" line with guaranteed <2hr delivery. They also let you reimage your servers through their control panel or API; you don't need to have a new one built just to get a fresh disk image.

You speak as though ops and automation is too mysterious for a startup to handle. There are so many tools and frameworks that do what AWS does that it's easy to acquire that expertise. And who says you always need a new server to iterate quickly?

I moved my last company completely off AWS and it was proven to be a great decision across a number of dimensions.

Can you list some of those tools and provide an estimate of how long they take to configure and how much day-to-day support they require?

Sure. Pop in opscode chef. Took me a weekend to write the basic framework, 3 weeks to make it solid. The 3 weeks more than paid for itself and hosting the config servers with them is a couple hundred a month. I could've hosted it myself too. This includes support for things like a load balancer, heterogeneous nodes (db, app, cache, chat, etc).

Ansible, puppet, sprinkler, and the like would take a similar amount of time to configure.

A bunch of Chef cookbooks does not AWS make. Configuration management tools are of course a necessity in AWS but do not replace their offerings.

I'm very impressed that you were able to build in 3 weeks time a low-latency multi-data center application with master/slave database failover, robust fault tolerant load balancing, and backups that can be restored in minutes with an API to control all of those services. That would normally take a senior team of engineers several months to accomplish and have it be of the quality and reliability of the services provided by AWS.

More likely is that you had a use case or a mindset that did not suit AWS very well and was easy to implement on your own. That's awesome and I'm glad you were able to find better value elsewhere. AWS is not for everyone, and is definitely quite expensive on the pocketbook.

had master slave, had fault tolerant load balancing, had backup scripts, tested restore procedures. also had node upgrade procedures, and more.

it was more than just chef obviously but chef + any bare metal host environment gets you a large percentage of the way there. Tacking on specific aws services like route53 when necessary works too.

That's a great accomplishment in such a short period of time. I'm glad that you were able to save time and money by using another service. Thankfully, there are tons of options out there to suit every business' use cases.

I don't really follow you here; AWS is infrastructure. So all the chef/puppet stuff has to happen anyway.

The benefit of AWS is that you have some immediate bootstrap, and simple auto scaling. This last is a killer feature. Being able to scale your caches and load balances silently, based on metrics, is a real time and money saver.

Sure. You need to have a level of scale to need that :) but when you do, AWS can have some good features.

When I said AWS I was speaking more about the entire ecosystem which includes a load balancer, databases with failover, snapshotting, backups, dns lookups, and more.

So use AWS or Linode for that specific use case. Great, that's its strength.

Once it's clear that the new server will be needed long-term, transition to dedicated hardware to save money and get better performance.

DigitalOcean - Same or better speed, 1/6 or less of the price of ECC.

DO has a very limited API, no ability to add additional storage without resizing your droplets, and has no firewall protection without iptables being enabled. DO has its uses (I'm migrating my personal server there presently), but it's not even close to being in the same market/caliber as EC2.

For some businesses, the huge AWS feature set (RDS, EBS, ELB, security groups, VPC, ELB, EIPs, etc, etc, etc) is more valuable than the bottom $$ line. For others, those features aren't worth the added cost, but hand-wavey "just use DO" or "Just rent physical servers from SoftLayer/Rackspace" is disingenous.

TL;DR AWS has value above and beyond simply hour-to-hour elasticity.

Hey, I've started to learn about devops and systems administration recently and I've learned a ton in this thread and this article, so thanks for that and thanks to everyone else.

But do you know of any good resources to learn about those two things? And I'm taking about basic devops, before you even start worrying about automating, and the things you would actually automate–because I don't know what it is I should be doing in the first place.

Things like what you should be doing right after you SSH into your server, how to make your server secure, how to use nginx, chmod'ing permissions of files correctly, and things I don't even know about.

Is there a One Month Rails or Michael Hartl's RoR Tutorial for devops/sys admin?

Regardless, thanks for taking the time to read this :)

AWS is PCI compliant. http://aws.amazon.com/compliance/pci-dss-level-1-faqs/

DigitalOcean and most others are not.

AWS's security and API is lightyears ahead of everyone else.

Yes, this is all true. But the question remains - how much is that worth to you? For some it will be mandatory. For others (particularly startups), not as much.

If said startup is storing any personal data, it better be important.

I don't see how that makes sense.

If all you need is a server that is up 24/7, rent it by the month. You don't need information to make an educated choice, since they are pretty much all cheaper than EC2.

With the additional caveat that you have tons of money sure. With 24/7 load, you'd probably pay a 10x premium to use AWS.

Reserved Instances really drive the cost down quite a bit, but yes, it's expensive.

> particularly when you are a new startup and don't have enough information (or capital!) to make educated server purchases.

I doubt there are many founders who are technically informed enough to know about Amazon Web Services, but don't know about the other big 3 (Digital Ocean, Linode, Rackspace). If you truly don't, then you must not be a tech company, and I have a hard time believing a non-tech company without any technical founders would even know about AWS.

I've seen places where they had an Amazon instance up 24/7 just sitting there idle, while they paid hundreds a month. Just because they occasionally needed a server for data processing. Someone told them that it was fast to spin up an AWS instance, so they did. But no one told them how to manage it, and the people doing the work were not very competent at system administration. So, they wasted thousands on idle time, just because no one told them how to snapshot their EBS volumes and terminate an instance.

More people know about AWS than you may realize.

DO, Linode, and Rackspace have lower bottom line costs, but a (much) smaller feature set which means more operational work. Especially when kicking things off, developer and operational time is often far more valuable than the cost of the servers.

Yeah, but it you start out using too much of the AWS tools, you're far more likely to get trapped in the AWS infrastructure and end up paying significantly more in the long term (which is what they want!). I'm not saying that AWS doesn't have useful features, but you need to appreciate the costs of these things before starting. If you need to spend a bit more time on devops in the beginning, then so be it. If you're starting a company, there had better be some good reasons behind using AWS, aside from "because faster iteration, developer time!".

Specifically - what AWS features do you find to be useful at the beginning? You seem to have some specific use-cases in mind. I'm legitimately curious.

Depending which tools you're using you'll have to do a bit of work to migrate out of AWS, but there tools are usually pretty standard stuff managed for you.

For example, RDS is just a database instance with management. You'd have to invest some time to replicate what AWS already does for you, but its not rocket science. The same for Elasticache, Autoscaling, ELB, and most of their other services.

This is off-topic, but because I've just recently started my foray into devops/systems administration do you know any lists of good resources to learn about those two things? I'd love to see a guide like this [1] except that's for analytics.

Things like what you should do right after you SSH into a server, how to make your server secure, setting up nginx, chmod'ing permissions of files correctly, and things I don't even know.

Regardless if you get back to me, I appreciate you taking the time to read this :)

[1] https://segment.io/academy/

Absolutely, great point! AWS isn't for everyone, and there can be lots of cases where it's cheaper to use dedicated hardware. Shop around before jumping in. I've added this as a new tip at the end of the article (crediting you of course). Thanks!