It depends on what you mean by "serverless". Serverless webapps? Sure, maybe a fair bit. But the real killer feature of the Lambda service is that it's a universal extension mechanism for other AWS services. Want to run some shoddy virus scanning whenever an object is added to a bucket for compliance reasons? That's a lambda. Want to manipulate Firehose records before they are sent to the sink? That's a lambda. Want to execute some crazy Python function from an Athena SQL query? You guessed it. That's a lambda.
So when you say "It certainly hasn't made my life as a developer easier" it means you haven't needed to do these bespoke gluing together of stuff, because "being able to do this somehow" is infinitely easier than "not being able to do it at all because AWS hasn't exposed an API for this use case".
I think the true geniuses are the ones who spent so much time trying to sell people on "Server Admin is hard, lets hire it out!" as well as building the billing and metering systems.
In a gold rush, screw panning, sell shovels (and pans).
This statement is a red herring. People don't flock to AWS because they want to use lambdas. Lambdas are adopted by AWS customers who already have their data in S3 or DynamoDB and/or already use one of AWS message brokers, and happen to stumble on a requirement for a background that runs asynchronously in response to an event, such as someone uploading a file to a bucket.
When that usecases pops up, the decision process consists of either adding a brand new service to be ran on ECS, add a background task on a service already being developed, or simply write the event handler and deploy it somewhere for AWS to run without us bothering about scaling and maintaining stuff. Writing and deploying the handler is an order of magnitude simpler to deploy and maintain, which happens to win over some technical discussions.
I started off with GCP, Cloud Storage and Cloud Firestore, but it was such a hassle and in the end, I couldn't even manage to run my Cloud Functions. The 10s cold start time was another non starter factor that led me to switch.
Honestly, I don't see myself staying with DynamoDB or AWS lambda. I definitely prefer local-hosted data stores such as on a DO droplet, and I would have started off with them were it not for their shoddy service. AWS service has been extremely strong so far.
Instead I suggested writing a Lambda service in Go and to run it incrementally (I think it was hourly and monthly). It worked out to tens of dollars per month, basically free. [At first it cost much more than this because the original implementation wasn't specifying known sizes for initializing collections that was constantly being resized in several places in its pipeline resulting in copying/double consumption plus severe fragmentation.]
The development, in particular versioning and deployment sucked back then. It's surprising to still hear mention of challenges so much later.
IAM (consistent identity and access control across all of your estate) and KMS (at rest encryption of all data, partitioned by service) are two examples of two things that would be very hard to replicate on your own hardware.
For example: Setting up different SSH keys when deploying a machine is not hard; getting granular ACL is more complicated.
Getting encryption at rest is very service specific, for example Postgres has TDE (free, with paid support if my memory serves correctly); I'm sure there are some services where EaR has been implemented by aws and there is no OSS alternative.
Base functionality is relative. The base functionality of IAM is that every server, pod, user or AWS service has an centrally managed identity controlled via RBAC that is used to access any other AWS resource, requires no long-lived credentials, powered via a flexible policy engine that can rely on principal and resource attributes, with updates to policies applied in seconds across the globe. The actual advanced functionality is more complex and useful in certain situations.
If your benchmark for base identity functionality is “different SSH keys for different servers” then I’m not sure what to say.
The same for encryption - the point is that it’s not very service specific with AWS. That’s the base functionality: encrypt this data (queue, disk, object, database, backup, whatever) with this key. Done. Consistent, transparent encryption at rest for every service, access controlled through IAM roles and policies.
It’s a hammer, and if it’s a good fit for your particular nail is very dependent on it’s shape.
That, and also offload some somewhat computationally expensive background tasks. Instead if screwing up your deployments by ramping up CPU and memory utilization to run a one-off task, just let he service fire a lambda and, if needed, just act when the lambda replies back.
Appropriate for a small amount of apps, but used inappropriately by a lot more for a while because it's the hot new thing.
And if one doesn't need an API first design then SSR with a robust framework might be cheaper to stand up and maintain.
The complexity bogeyman is a red herring. What is software development if not a constant fight between increasing complexity and adding features, including making systems more robust and resilient and reliable?
Sure, you are free to claim that function-as-a-service paradigm is very complex because the code runs somewhere else, and you have to use a separate workflow than the one you're used to, and that you actually have to write unit tests to have some assurances that stuff does work as expected.
But what would be the alternative approach?
Would it be simpler to write a full-blown service that polls events and listens to message queues and launches background tasks? Obviously, no.
AWS Lambda is a fancy way to let developers develop event-driven systems by just writing tiny event handlers and plug in events. It's no rocket science. In terms of complexity, they pale in comparison with even the tiniest hello world windows GUI app, or even Qt. I'm terms of web services, they are a service where you just have to write the request controller bit after that runs after all auth things passed and request messages were parsed and validated. Considering this, isn't it unreasonable to talk about increasing complexity, when everything was made.far simpler than what it would otherwise be?
It's not really a red herring when Lambda mostly forces communication between different application layers to be over the network. You suddenly have to deal with rate limiting, retries, bulkheading, etc. in a lot more places than you would even with even a SOA.
> you actually have to write unit tests
I'm not really sure unit tests are a differing factor in testing functions as a service and other software. The real issue with testing is that it's very difficult if not impossible to do more than unit testing locally. You're often using docker containers or mock services to try and simulate what is going on in production which is insufficient. Things like CloudFormation can't be done locally (and this can be where a lot of gross complexity lies), so ultimately you have to set up an identical-to-prod testing environment and do the bulk of your testing there (which you probably should have anyway, but that's besides the point). Things like localstack HAVING to exist but not being supplied by AWS is a symptom of something gone wrong.
> Would it be simpler to write a full-blown service that polls events and listens to message queues and launches background tasks? Obviously, no.
Why is this obviously no? It isn't to me. You likely need a lot less these type of things if you're running a monolith.
> In terms of complexity, they pale in comparison with even the tiniest hello world windows GUI app, or even Qt.
It seems to me you're ignoring all the extra stuff you HAVE to have to deploy, secure, and maintain serverless. There is a lot more than the code for the function themselves.
Well now you're deep into Terraform or CloudFormation, setting up API Gateway, building an authorizer, coming up with an API Schema, wiring up everything with IAM, setting up Route 53 and Certificate Manager, building CloudWatch Dashboards, etc etc. I hope you got all the permissions right because there are now 25 different places someone can maliciously gain access to your AWS account. And in order to change anything you should be prepared to update your IaaC schema once every couple months or it will all stop working.
Back in days past you'd rent a hosted server or shell account (It was hideous, but bear with me), write 15 lines of PHP, connect to the included database, and you're done. It wasn't "web scale" but the time to reach a MVP was probably 1/100th that of the current era of gross complexity. And any one who built software in that time period had a very clear runway to bring it in house and run on bare-metal. No such thing exists in today's era of subscription-model infrastructure.
No, not really. You only use any of that if that's what you want to do. You can use a EC2 instance and do everything yourself, if that's your thing.
I mean, let's go through your list. CloudFormation is just infrastructure as code. It helps treat your deployments as cattle, but you aren't forced to use it, right? And you only need an API gateway if you want to put together an API using Amazon's proprietary stuff. Nothing forces you to do that, right? And you only need Route53 if you feel the need to manage a DNS and do stuff as DNS-based routing. And dashboards is just to show metrics in a way that fits your need.
From these services, which ones are a must-have? None.
> Back in days past you'd rent a hosted server (...)
Your "back in the day" is "right now" if you want to. You can fire up an EC2 instance, open a port, and done. Why are you leaving this out, and instead opt to misrepresent extras as required?
Hell, AWS also offers lightsail, and you can use S3 to run static sites directly. Both are far simpler than managing your own VM or baremetal service. Why are you leaving that out?
And why do you leave out the fact that with API Gateway and lambdas you can setup a fully versioned API with multiple endpoints in a few minutes? You don't even need to deploy anything as you can write the controllers for each route directly in the dashboard. If reaching an MVP instantly is what you really want, it's weird you leave that out.
> No such thing exists in today's era of subscription-model infrastructure.
I regret to tell you that you don't know what you're talking about. Even in AWS, where they try to upsell you everything, you can easily ramp up a VM, open a port, and do everything you need by ssh-ing into a linux box. And AWS is not alone on it. You can do the same on pretty much any cloud vendor. Say Hetzner: you can rent a baremetal box for around 30€ a month and leave it at that. If you prefer VMs you can get one for 3€ month. What's stopping you?
The same thing happened on frontend: all the juniors bootcampers' only experience of doing frontend development is running create-react-app and hacking away at React. Before that it was angular.
Everybody would frown at deploying a pure frontend js app or a jQuery app, in the same way most people would frown at deploying a PHP script on Apache.
Despite this, a huge portion of the internet runs on PHP and jQuery (including shiny new things, like levels.io's million making 1-2 men saas).
Terraform is a magical amount of arcane.
Static sites do not involve any form of tasks being executed: you just serve dumb files and let everything happen client side. I fail to see how a usecases that involves zero computational needs, and arguably zero servers, is used as an example in a discussion about how to implement services.
Static site? Yes. Basic CRUD? Not sure about cost efficiency, but otherwise that’s right in the APIGateway+Lambda wheelhouse.
In my own work with serverless I think about this a lot. In my day job, we built a new service and we pay AWS a few dollars per month to run. It would have cost us around $100 a month in AWS costs. However, the operational complexity is extremely high and we are also coupled to another service via Kinesis. For a billion dollar business, the trade off doesn't seem worth it.
I wonder how much of serverless is just AWS firing at Google (and Heroku, DO, etc) and other competitors. It certainly hasn't made my life as a developer easier. It's certainly much cheaper for some use cases but the complexity of the system goes up very quickly, and you're end up having to manage a lot of that complexity using an inferior tool like Cloud Formation (or Terraform).