So a lot of effort goes into producing a site that could be "Facebook scale" if it became necessary. Although since this is difficult (and expensive!) to test, it's likely that some panicked manual intervention will be required if it does take off.
It's really worth asking "what is the cost of scaling" next to "what is the cost of NOT scaling, and what are the chances of it happening?". Is your organisation even able to benefit from rapid scaling - would it be profitable? Or are you going to end up like that luggage company screaming at its customer service reps to work dozens of hours a day?
If your cloud hosting cost goes from free-tier to $100,000/month in one month, and it's real traffic, is your business actually going to be able to pay that invoice?
Is the static overhead of scalability better for your use case than what a normal PC is capable of, which these days can be quite a lot: https://adamdrake.com/command-line-tools-can-be-235x-faster-...
(Been thinking about this since I mentioned the SRCF earlier, which was a single-server host for years: https://news.ycombinator.com/item?id=21741789 )
The cost of running a production server is something small startup businesses need to budget for... And you don't want to run your production website on a 5-10/month shared server. If you design for the cloud/serverless, you can get your costs down to near-$0 for light usage... But yes, using things like lambda mean a possible bill shock if your traffic grows faster than your income can recover it.. but if you have an advertisement on the page, it probably will cover the execution cost if it's lean enough.
In another example, I inherited a CICD pipeline that cost $300/month and could run 2 of our larger test suites simultaneously.
I converted this to AWS Spot instances and now support thousands in parallel and I pay under $170/month ( https://ldoughty.com/2018/01/gitlab-multi-runner-w/docker-ma... )... But to your point about "what if" costs of this, yes, there's a limit on place for this setup so it can't go over 20 or so... And it is important to consider the impact of auto scaling with regards to the bill... For a small business, AWS effectively doesn't have a way for you to really stop "Bill attacks" or legitimate high traffic on API Gateway/Lambda short of you taking down your site.
My recommendation is: ideally, don't expose something that costs you much too run unless you can recoup the cost from that action (paying customer, advertising).. your homepage should be static or cached in a CDN until you're ready for that large bill.
Multiply that by the amount of money you cost your employer per hour and you'll realize that your "optimization" actually resulted in a net loss.
So by that, I've saved $5,000 in pure server costs. It probably cost me 10 extra man hours to do this way... But I also resolved a bug that was not terribly worth fixing before but took up 30 minutes of time every few months.. and improved the security by taking the servers off the public web.. however you want to quantify security improvements.
I always look at opportunity costs and the automation cost/benefit! The article was researched and written on my own time for my own enjoyment (so this was actually "free" from a business perspective, but my responses factored it in as if I did it as paid work. Just clarifying in case someone wants to claim I'm publishing "work"
That said, it really wasn't much work to drop the 6pm-6am costs to effectively 0 unless a developer decided to work late (it automatically scales up)
Google's Firebase Functions are ideal for REST endpoints, they bill by the millisecond instead of AWS's 100ms (I think?) pricing.
If you buy into Google's ecosystem entirely they provide you a lot of functionality to get off the ground running. E.g. their (free!) auth service integrates with security around functions. In less time than it takes to look up the YAML for Nginx config you can have a proper authenticated set of REST endpoints up.
Now that said all I have my CPU intensive stuff running on a fixed price VM.
> The Aurora Serverless failover time is currently undefined […]
Don't get me wrong: I really like the idea of Aurora Serverless, but with the current feature set it isn't very compelling.
[1]: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...
Because Aurora separates computation capacity and storage, the storage volume for the cluster is spread across multiple AZs. Your data remains available even if outages affect the DB instance or the associated AZ.
- Data is replicated cross region, but your compute all lives in the same AZ, so if (when) an AZ goes down you have to wait for a new instance to be created elsewhere. AWS says the time for this is undefined, IIRC we've seen it be around 15-20 minutes before the DB is back online.
- If you have lots of long running transactions or queries, serverless can have trouble finding a scaling point and won't scale up/down. You can set it to force scaling after 5 attempts, but this results in dropped connections and 1-2 minutes of downtime every time.
- Scaling up actually takes 45s-1:30 for new capacity to be available. If your load is spiky enough that that's too slow, you're stuck with overprovisioning anyways.
- Tools like VividCortex don't work for serverless if you rely on those. Teams here that use serverless have shifted to DataDog APM for this purpose.
- Loading data from S3 doesn't work. This wasn't really a usecase we had but it's something to be aware of.
That said, for gradual load or DB downtime tolerant services it's great! Also very nice for dev environments as the scaling down to 0 can result in some very real cost savings.
If you're in a traditional three tier architecture you might run into your database being overloaded by the autoscaling behavior but this is fundamentally a provisioning problem. These problems are better served in models that are scalable.
Your suggestion to by a (necessarily overprovisioned) on prem database server is cost inefficient, degrades in value over time, and is effectively a lock in.
Embracing eventual consistency is not that scary!
And nobody said anything about on-prem databases. You can hit these issues just as easily with e.g. Aurora or Dynamo.
I do wonder what limitations and caveats you are talking about. That is true for many databases who are only consistent on one row/collection/region/snapshot, have strange limitations such as no joins, require your data to be preformatted in a certain way etc. DynamoDB also has a few limitations in those aspects. I have both love and hate memories from my experience with DynamoDB. But nowadays you get to choose between databases like FaunaDb, Spanner, FoundationDB, CockroachDB... So what are the Caveats you still worry about?
If you would combine such distributed databases with edge serverless frameworks like Cloudflare workers I could see amazing possibilities tbh. It's strange how front-end programmers seem to be picking up distributed databases at great speeds due to the benefits of low latency for their users, zero configuration and autoscaling. But it seems that we backenders are reluctant and still prefer our hardware for some reason :). Is it pure pricing?
We have another process that is read intensive. The database is a constraint. We autoscale read replicas. With AWS, all of your read replicas read from the same storage arrays (grossly simplifying)
It seems to me, like most things in software these days, we sacrifice performance for convenience at every turn; and in the cloud case specifically we also pick up a good portion of complexity which rarely pays dividends.
I'd wager that for most non-global scale products you can spend a few thousand dollars at the local PC shop and assemble some hardware that will more than support your needs. Although, I'd still advocate backing up necessary data in the cloud for resiliency.
If your software is super resource intensive then you could have people install and run it locally gasp.
That sums it up pretty well.
The great irony here is that those thin, stateless web servers that are the only things to easily autoscale are pretty much guaranteed to never be your performance bottleneck, unless you're doing something really strange.
all you have to do is run Wordpress with Woocommerce, multiple additional plugins and an overloaded theme with visual builder/editor that needs 512 megs of RAM and 500ms to display a contact form[0]. Some themes can be amazingly CPU intensive.
[0]and 900+ queries, but since most of them are simple selects any cache like redis or memcached removes their performance impact.
How is this true? Cassandra/MongoDB/Scylla are all very very popular NoSQL DBs that scale very well and work for a good number of common use cases. Sure you have to be a little more clever when writing certain types of applications with them, but seems like an appropriate trade off. If you don't like that trade off, you can spend a bit more money and get Spanner/FaunaDB and you don't have really any tradeoffs (other than $$$).
Say you have an idea for a web app that needs a server-side component, but you only ever expect to have a few users. A VPS will cost you at least $5 a month. If you can make your backend serverless, you'll probably only pay a tiny sliver of that, if you even end up exceeding your provider's free tier.
In those scenarios, the _real_ cost isn't the $5/mo VPS but the managing upgrades, security patches, etc - which would be mostly handled for you in this scenario.
I built a simple email verification service for A company I used to work for. Used API gateway, lambda, s3, and sqs. Few thousand users a month. I don't need to maintain it, check for security updates etc.
The monthly bill for all that was ~$2 NZD.
I run a bunch of stuff on a $5 VPS and more still on small machines like raspberry pi's and so on behind it.
On my distro at least, spinning up a real paravirt container (not docker, a full OS with systemd, SSH, etc) within a VPS is a matter of unzipping a bootstrap to /var/lib/machines and running 'machinectl start whatever-you-called-it'.
It's just a different set of abstractions to learn, and at least IMO one that's more valuable.
For most applications, it's really not needed though.
In terms of the actual technology itself, it's very interestingly built, and as the poster above me mentioned, with a ton of caveats to the promises it makes.
I hate and love it at the same time.
This does not seem scalable for OLTP type load on some busy store. Again I think you'll be way better of money wise hosting your own database on real hardware either on prem or colocated
"Across the 48 hours of Prime Day, these sources made 7.11 trillion calls to the DynamoDB API, peaking at 45.4 million requests per second." [1]
[1] https://aws.amazon.com/blogs/aws/amazon-prime-day-2019-power...
my understanding is that most storage solution scales well when planned, but scaling during a spike only causes more stress onto the current replicas/partitions for all the time needed to boot and shift data toward new replica/partitions, and the new instances don't contribute in moving the load until the process is complete.
so they work if you can predict a spike, but they can't handle a sudden spike, especially if the data to replicate/partition is sizeable.
I am always puzzled about this "autoscale" thing on a cloud.
If your task can be represented as something like calculate sum of some ginormous array then sure. Split array in parts and launch thousand instances each working on it's own slice and then combine.
In way more common situation you have a service hitting database and doing something with it. Sure you can spin a thousand instances of said service. But they will all be hitting the same database. So in reality I think the only really scalable part here is the database. And scalability in databases has no universal solution and the ones that exists come with the load of caveats.
I think unless you operate on FB/etc scale the simplest and cheapest solution would be to buy/rent nice piece of hardware yourself and place database on said hardware. That should give you a nice run for your money and free you from bazillion layers of abstractions, dependencies, etc. that often tend to fall apart.