In a cloud environment - which is relevant since he was using AWS - it’s much cheaper and easier to autoscale app servers than database servers based on load. It could also be conceivable cheaper if you’re doing non time sensitive online analytics processing instead of something that requires a real time response. On AWS, you could set autoscaling and use spot instances to scale more when the rates are cheaper for compute.

Also, when scaling up a database, there is more of a potential for wasting money because of idle capacity when the resources aren’t needed. It’s painless to remove an app VM out of service.

Yes I know you can also use DynamoDB, but it’s non relational, the cost of adding additional global indexes and paying for extra read and write capacity gets expensive.

There’s also Serverless Aurora that auto scales but it would also get really expensive fast.

Of course, in that case, one is so limited by Amazon's specific hardware choices, if not by virtualization [1]. The OC didn't specify AWS but did mention MySQL, instead of Postgres, which further limits ones ability to implement basic performance optimizations, like separating data and index I/O [2].

> it’s much cheaper and easier to autoscale app servers than database servers based on load

Yet I'm still not convinced this is true if the "load" is actually I/O-bound rather than CPU-bound [3]. It may be easier to scale (especially autoscale), but, with cloud providers, that doesn't necessarily mean cheaper. In practice, distributed computing never scales as good as linearly, and people tend not to even measure for diminishing returns or how Amdahl's Law affects them.

> It could also be conceivable cheaper if you’re doing non time sensitive online analytics processing instead of something that requires a real time response.

I believe GCP has instituted single-instance pricing that's essentially pay-what-you-use in terms of number of cores, so that's even possible with a database server on that provider, but, again, only if the issue is CPU, which I maintain it generally isn't.

> Also, when scaling up a database, there is more of a potential for wasting money because of idle capacity when the resources aren’t needed.

I recall this sort of thing being the original argument for virtualization, when it first came out, but it only held water if one was grossly over-paying for the resources to begin with (which most "enterprise" shops certainly were, and still are).

To me, it's an argument for why, beyond modest size, buying cloud infrastructure is just over-paying.

> paying for extra read and write capacity gets expensive

> Serverless Aurora that auto scales but it would also get really expensive fast

Par for the course for any cloud services, in that they may seem convenient or "auto scale" but the price premium is tremendous, especially at scale. How do they compare with paying for that capacity over EBS?

[1] Yes, I know they now have "bare metal" instances, or is it still only instance (singular)?

[2] Not that it matters much if one is limited to 10Gb/s or even 40Gb/s (if you can get it) EBS volumes.

[3] Which the OC's sort very likely would be but the OP's join would not. To that extent, perhaps the OC's whole premise is a red herring, because I doubt anyone is suggesting not offloading to the app an ORDER BY that's just the last step before presentation to the user.

Like a lot of things dealing with autoscaling - lambda, Fargate (serverless Docker), serverless Aurora - autoscaling saves money when your demand is really spikey and even then you have to do a cost comparison.

For Linux instances, you get per second (or minute?) billing so the cost of bringing instances up and down isn't that expensive. For Windows instances, you are always billed by the hour.

* [2] Not that it matters much if one is limited to 10Gb/s or even 40Gb/s (if you can get it) EBS volumes. *

The fastest throughput that I am aware of with EBS is 500MB/sec without doing RAID (https://aws.amazon.com/about-aws/whats-new/2017/12/amazon-eb...)

Of course instance storage is faster (https://gist.github.com/ktheory/3c3616fca42a3716346b), but is that a thing people do for databases?

Indeed. I've found the spikiness ratio (if that coined term makes sense) to be 2x-10x for AWS in general, and I was wondering if you (or anyone else) had a sense of how much that increased for these more premium services.

> and even then you have to do a cost comparison.

I have rarely seen this done, and, when I've talked to people attempting it, the common complaint is that AWS doesn't make it easy.

> The fastest throughput that I am aware of with EBS is 500MB/sec without doing RAID

Yes, per volume. I was, perhaps, too terse in my footnote. I was referring to the maximum throughput per EC2 node. Considering 40Gb/s would be saturated by about 10 of these, that's not very much. Real hardware can handle quite a bit more than 10 such SSDs attached.

> Of course instance storage is faster (https://gist.github.com/ktheory/3c3616fca42a3716346b), but is that a thing people do for databases?

I would hope so, but probably not in general practice, as it would require too much specialized knowledge. The main problems are that it's ephemeral and very limited in quantity. The impact of the former is reduced by the latter, of course.

I’m first and foremost a developer, I play the role of the “resident certified AWS Architect” when necessary. But the only time I concern myself with cost is when the CTO - who is also pretty knowledgeable about AWS - calls me into his office to complain....

For all the lip service given to "you have to do a cost comparison", it doesn't happen in practice.

The issue is, there are relatively few people that can run a data center well compared to the number of people who need servers to run thier business. The cost of hiring people to manage local servers is higher. For a small company, you can get away with one or two devs who also have AWS training and outsourcing your heavy lifting to an APN that charges around 10% of your monthly AWS cost. They usually outsource the grunt work overseas or do "rural sourcing" if regulations don't allow overseas access to the data.

Also, you don't spend as much up front. If you are running a well run business. Your expenses on AWS should scale a lot slower than your revenue.

I don't think that's true, other than as a result of a self-fulfilling prophecy.

> The cost of hiring people to manage local servers is higher.

That, there, is the falsehood at the root of the prophecy. I keep reading comments on HN of the form "and then we'll have to hire 5 sysadmins for those 200 physical servers" but they're invariably from people who have done nothing of the sort.

From personal experience, the incremental effort for dealing with the physical part takes up around a quarter of a full-time Ops person's time at that scale [1]. Even at high salaries, that's less than any cloud premium. Perhaps more importantly, the vast majority of those skills aren't difficult and could easily be imparted onto a lower-cost employee. Of course, the idea of providing mentorship or career paths for junior employees is anathema in today's environment, especially in smaller companies.

None of the other Ops labor costs go away, and, judging by salary ranges in SFBA job postings, AWS-expert Ops folks are 10-15% more expensive.

> Also, you don't spend as much up front.

I routinely hear and read this argument, as well, but, given the huge price premium of AWS [2], it doesn't withstand scrutiny.

If the price premium is 6x [3], that means paying the equivalent of the "up front" cost to AWS in 6 months. It might just feel emotionally easier to have it spread out over half a year rather than having to write a big check all at once.

All that said, I wouldn't advocate an early startup (or other small company) doing anything but cloud or VPS.

> Your expenses on AWS should scale a lot slower than your revenue.

The issue is, there are relatively few people that can run AWS well compared to the number of people who use AWS to run their business.

[1] and tends to scale very sub-linearly, often logarithmically if there's low diversity in model/configuration, with number of servers

[2] GCP has slightly better pricing and some VPS providers are remarkably better, but I posit cost (up front or not) is never a real consideration and so these aren't even brought up in conversation

[3] For various workloads, my analyses have fallen in the range 2x-10x, assuming 3 year contracts, 3 years of use out of the hardware, and zero residual value. Yes, that's all-inclusive, even of my own labor. 6x for just the hardware (up front) portion is about right for typical workloads.

But you also have the turn around time to actual get the physical boxes and you can’t run experiments as easily. Just the other day I was working on a project to see how much throughout I could get processing a queue of messages based on the number of instances, the optimal size of the instance, the correct criteria for scaling in and out before other constraints started hitting us. I kept playing around with it rerunning the Cloud Formation script with different parameters until I got my answer at one point I got up to 20 8GB 4vcpu instances. We settled on running one instance all of the time up to 8 based on load and just for grins and giggles I set up a separate ElasticSearch node just to run analytics. Can you imagine the red tape and the time it would have taken to do that self hosted? We also would have been paying for 14 (two similar processes that work on different queues) to sit idle most of the time just to handle the load.

I also would have had to provision VMs for ElasticSearch and Kibanna, RabbitMq, and some type of alerting system. All this would have needed to be redundant across two data centers for DR once we moved into production.

I didn’t even mention the build server that would sit idle most of the time. CodeBuild is also “Serverless”. You either use one of the prebuilt Docker images or use your own and only get charged for use.

* Even at high salaries, that's less than any cloud premium. Perhaps more importantly, the vast majority of those skills aren't difficult and could easily be imparted onto a lower-cost employee.*

You can’t get too much of a lower cost than outsourcing your netops to a separate company that charges around 10% of your monthly bill. They then outsource to either cheaper foreign labor or to some place in middle America.

Of course, the idea of providing mentorship or career paths for junior employees is anathema in today's environment, especially in smaller companies.

Who is going to train the trainer? It wasn’t until 4 years in my company that we had any onsite AWS expertise. They outsourced support.

None of the other Ops labor costs go away, and, judging by salary ranges in SFBA job postings, AWS-expert Ops folks are 10-15% more expensive.

They don’t go away, they get spread out among multiple customers.

We spend very little time doing “netops” and a lot of the things we pay our APN for, we now have the expertise to do ourselves.

In my experience, an “AWS Architect” who doesn’t understand the development and devops side will do suboptimal “consulting”. The consultancy company never suggested, better, cheaper, more efficient ways to use AWS because all they knew was netops. The developers who have really gotten into AWS can go back and forth between netops, devops and development and can build much more efficient systems. Of course, you still need that one “benevolent dictator” that really knows their way around AWS (raises hand).

> Also, you don't spend as much up front.

I routinely hear and read this argument, as well, but, given the huge price premium of AWS [2], it doesn't withstand scrutiny. If the price premium is 6x [3], that means paying the equivalent of the "up front" cost to AWS in 6 months. It might just feel emotionally easier to have it spread out over half a year rather than having to write a big check all at once.

If you’re paying six times the cost, I would suspect....

1. You’re not taking into account the total cost of an equivalent redundant system within a data center and between geographically redundant data centers (OMG our load balancer went down in the middle of the night).

2. You’re overprovisioning and not architecting with the mindset of elasticity.

3. You’re not taking into account the ability to move a lot faster by not having the lead time between an idea and provisioning hardware and getting things to market. The whole project I spoke about earlier that included 16 VMs at maximum load with a total 64 hyperthreads, 128GB of RAM, separate servers to run ElasticSearch, a performant queueing system, some type of monitoring system, etc. could never have been stood up in a week.

That’s not even to mention that we had to duplicate the same setup in a Dev environment - which could be done on both Dev and production by just running a CF script with different parameters.

My CxO came to me and said we need to prove that we can handle this load before we can get this contract. It took me less than two weeks to create the entire architecture - that included some rework on the programs. I didn’t have to work with netops and devops who then had to work with the finance department who had to worry about what would we do with all of this hardware we provisions just as a proof of concept.

Hypothetically, if we moved on to something else and we didn’t need the resources from the project, I would just delete the Cloud Formation stack and all of the resources are gone - as opposed to having all of this hardware sitting around.

4. You’re not taking into account either a global workforce or global customer base. At one company we had people working overseas, performance was horrible.

We created another account, duplicated our Dev infrastructure using CloudFormer and put everything in thier region. Of course I didn’t want to deal with it. We paid our Amazon Partner for a one time bucket of hours. Even our netops and infrastructure support is “elastic”.

The issue is, there are relatively few people that can run AWS well compared to the number of people who use AWS to run their business.

I blame “AWS Architects” who take thier on prem netops mindset to the cloud, developers who don’t know how to architect with elasticity in mind, and devops folks who don’t know how to use AWSs built in devops tools. There is no reason that costs shouldn’t go up linearly (at a lower slope) as the number of customers increase.

[1] and tends to scale very sub-linearly, often logarithmically if there's low diversity in model/configuration, with number of servers

If you have a stable workload, with predictable growth, and you’re a typical large enterprise, the cloud may not make sense (see GE’s Predix platform).

Also if you are a low margin B2C company it may not make sense at a certain scale.

That's a false dichotomy. There's nothing saying that, when one has physical infrastructure, that one cannot also use cloud infrastructure in cases where it's needed (and transition when it's not.. it's the transition part that's usually missing).

> Can you imagine the red tape and the time it would have taken to do that self hosted?

I don't have to imagine, as I have experience doing things self hosted. "Red tape" is just a political problem within a company, not a technical one. In some cases, even time is zero, if I happen to have spare hardware to lend.

Regardless, the whole experimentation situation is a strawman. It is, effectively, indistinguishable from early startup.

> 1. You’re not taking into account the total cost of an equivalent redundant system within a data center and between geographically redundant data centers

Nope. Always apples-to-apples. Physical DC redundancy costs something, but it's approximately linear. (Double for twice the price). Multi-cloud gets very costly in engineering time, too, if you want true redundancy in that context.

> 2. You’re overprovisioning and not architecting with the mindset of elasticity.

Overprovisioning goes against the definition of comparable. The "mindset of elasticity" is entirely too hand-wavy, as it's only a savings mechanism, and only for part of the infrastucture. It also only works if the workload itself is actually elastic enough.

That's part of why my range for the AWS premium is 2x-10x. The other part is that not everything is about CPU+memory (i.e. EC2), and I/O, including EBS and data transfer costs can play a big role.

> 3. You’re not taking into account the ability to move a lot faster by not having the lead time between an idea and provisioning hardware and getting things to market.

Besides being a strawman argument, that has nothing to do with the infrastructure cost. After bridging the initial lead time, AWS will still cost multiple times more, over years, if not transitioned off of.

> 4. You’re not taking into account either a global workforce or global customer base. At one company we had people working overseas, performance was horrible.

Again, nothing to do with the cost of the underlying infrastructure. It's just an argument of the form "<benefit of cloud> makes it worth the premium price", which isn't the same as the premium not existing.

I'm willing to concede that, for certain location, AWS might actually be cost-competitive, due to, for example, otherwise insufficient competition in commodity hardware vendors or internet service. However, I'm skeptical that such a place exists and is served by AWS.

> I blame “AWS Architects” who take thier on prem netops mindset to the cloud,

I'm not sure how you can legitimately blame the existence of those you consider incompetent for the non-existence of those who are competent.

> developers who don’t know how to architect with elasticity in mind,

I'd argue there's nothing wrong with that, as they shouldn't have to care. Plus, some workloads just aren't easily elasticized, just like some (many) aren't easily parallelizable with software, no matter how much you might personally wish it were so. That not-easy can translate to huge engineering costs (which I've seen for parallelization, though not elasticity, for the very reason you mentioned).

> and devops folks who don’t know how to use AWSs built in devops tools.

That sounds like a recipe for vendor lock-in, which is another way of saying higher-cost-lock-in.

> There is no reason that costs shouldn’t go up linearly (at a lower slope) as the number of customers increase.

There are plenty of reasons, not the least of which is naive/inefficient distributed systems design. Notwithstanding the fact that No True Scotsman would ever create such a system, I've witnessed more than a few.

> with predictable growth

Everyone has predictable growth, given a short enough timescale. That's part of the point of using commodity hardware, rather than something like "enterprise" hardware, where sometimes just the sales cycle can be prohibitively long.

Extrapolating lead times from ones experience in a giant corporation or from the dot-com era is unrealistic.

> Also if you are a low margin B2C company it may not make sense at a certain scale.

I posit that neither the size of the margin nor the nature of the business makes any difference. Having a vastly too high COGS doesn't make sense for any business.