Connecting to AWS managed services (s3, kinesis, dynamodb, sns) don't have this overhead so you can actually perform some task that involves reading/writing data.

Lambda is basically just glue code to connect AWS services together. It's not a general purpose platform. Think "IFTTT for AWS"

It’s a standard CRUD REST API used by our website.

The only thing slow is infamous cold start time when running within a VPC because it has to create an ENI.

AWS pinky promised they were going to fix this soon.

If I put the node service and a database on the same box I'd get the same performance, and actually probably better since Amazon would still have them on separate physical hardware.

If you use open source interfaces, or even proprietary interfaces that are portable, it's easier to take your app with you to the new hosting provider.

The non-portable interfaces are crux of the matter. If you could run lambda on Google, Azure or your own metal, folks wouldn't feel so locked-in.

But, I could still take advantage of hosted Aurora (MySQL/Postgres), DocumentDB (Mongo), ElasticCache (Memcached/Redis) or even Redshift (Postgres compatible interface) without any of the dreaded “lock-in”.

My position isn't don't use AWS as a hosting provider, it's that you ought to avoid being locked into a proprietary non-portable interface when possible.

Or the Twitter model - very bad architecture that always crashed, find “product market fit” and then get funding to fix any issues.

Or the company goes out of business, I put X years of AWS experience on my resume and make out like a bandit as an overpriced consultant.

I don’t see the downside....

https://aws.amazon.com/blogs/aws/firecracker-lightweight-vir...

If writing code to be able to move to a different cloud isn't considered lock-in, then nothing is since anyone can write code to do anything themselves.

There are two kinds of lock-in: high cost because no competitor does as a good a job - this is good lock-in, and trying to avoid this just means you’re not building the thing optimally in the first place.

There is also high switching cost because of unique interface and implementation requirmenrs that don’t add any value over a more interopable standard. This is the kind that’s worth avoiding if you can.

"Connecting to AWS managed services (s3, kinesis, dynamodb, sns) don't have this overhead so you can actually perform some task that involves reading/writing data."

That is due to network and colocation efficiencies. The overhead of managing such services yourself is another matter.

I’ve done stress testing by spinning up and tearing down multiple VMs played with different size databases, autoscaled read replicas for performance. Ran a spot fleet, etc.

When you need things now you don’t have time to requisition hardware and get it sent to your colo.

And this doesn’t count all of the third party hosted services.

Aurora (Mysql) redundantly writes your data to six different storage devices across multiple availability zones. The read replicas read from the same disks. As soon as you bring up a read replica, the data is already there. You can’t do that with a standard Mysql read replica.

You connect to S3, and:

a) You can build an abstraction service if you care about vendor lock-in so much

b) It has an API that plenty of open source projects are compatible with (I believe Google's storage is compatible as well)

Maybe you use something like SQS or SNS. Bummer, those are gonna "lock you in". But I've personally migrated between queueing solutions before and it shouldn't be a big deal to do so.

It's really easy to avoid lockin, lambda really doesn't make it any harder than EC2 at all.

As I said, at least in the cases of your database and your storage, being cloud-agnostic is trivial. Managed postgres is easy to migrate from, S3 shouldn't be hard to migrate from either.

Certainly lambda doesn't impact this too much.

> Serverless technology is only starting out still and I'm pretty sure 5 years from now Lambda won't be the go-to platform anyway. Plus honestly we've learned so much from the first big project on Lambda that writing the next one with all of that in mind will be pretty great (and agnostic).

I realize it isn't entirely on-topic, but could you elaborate? I'm curious to hear more about your opinion on this, I'm not sure what the future of Serverless is.

Heck, consultants get paid lots of money just to do a lift and shift and migrate a bunch of VMWare images from on prem to AWS.

Yes, you can build an abstraction layer. And maintain it. And hope that you don't get feature divergence underneath it.

That's really, really expensive.

I’ve had to explain to a CTO before why I had my team spending time on a CI/CD pipeline. Even now that I have a CTO whose idea of “writing requirements” is throwing together a Python proof of concept script and playing with Athena (writing Sql against a large CSV file stored in S3), I still better be able to articulate business value for any technological tangents I am going on.

Any time spent on work that doesn't directly create value for customers is work that the business should be weighing in on. I'm not saying that you should never spend any time doing anything else - but these are trade offs that the product manager should be involved in, and one of their primary jobs is being able to weight the technical and business realities and figuring out where resources should be going.

My only point is that vendor lock-in is not a significant issue on AWS, and that it requires virtually no effort to avoid it.

Of course it requires effort. A lot of effort, not to mention headcount. The entire value of cloud-managed services is what it saves you vs. the trade-off's, and it's disingenuous to pretend that's not the case.

There are tradeoffs, of course. Cost at scale is the really big one - at some point it's cheaper to bring ops/ hardware in-house.

I just don't agree that lock-in is a huge issue, and I really disagree with the idea that lambdas make lock-in harder.

- DBA's & DevOps

- Procurement management & spare parts

- Colocation w/multihoming

- Leasing agreements

- Paying for power usage

- Disaster recovery plan

- CapEx & depreciation

- Uncomfortable meetings with my CFO explaining why things are expensive

- Hardware failure

- Scaling up/out

Not even worth going on because the point is obvious. Going "all in" reduces cost and allows more time to be focused on revenue-generating work. The "migration" boogeyman is just that, something we tell other programmers to scare them around the campfire. You're going to be hard-pressed finding horror stories of companies in "cloud lock-in" that isn't a consultant trying to sell you something.

> at some point it's cheaper to bring ops/ hardware in-house.

It depends. It's not always scale issue, and with all things it starts with a model and collaboration with your finance team.

Using a company that bypasses Amazon for 99.999% of its traffic isn't exactly an Amazon success story.

Everything was built on Amazon and video is largely hosted on S3. Yes, there’s a large CDN in the mix too. That doesn’t take away from the achievement.

https://www.google.com/amp/s/techcrunch.com/2012/06/04/netfl...

Except for some proprietary light AWS proxy code, the bulk of the Lambdas delegate to pre-existing Java POJO classes.

The cold start issues and VPC configuration were a painful learning curve, but nothing I would consider proprietary to AWS. Those are universal deployment tasks.

This is false. I've seen entire Lambda APIs backed by MySQL on large, consumer-facing apps and websites. As another poster pointed out, the cold-start-in-a-VPC is a major PITA, but it can (mostly) be worked around.

Or is it when deploying new code?

"Why are you throwing rocks at that machine?"

"It makes it respond more quickly to actual client requests. Sometimes."

"Sometimes?"

"Well, most the time."

"Why's that? What's causing the initial latency?"

"Cold starts."

"Yeah, but what's that mean?"

"The machine is booting or caching runtime data or, you know, warming up and serverless. Anyway, don't think about it too much, just trust me on this rock thing. Speaking of which, I got to get back to bastardizing our core logic. Eddie had great results with his new exponential backup rock toss routine and I'm thinking of combining that with a graphql random parameter generator library that Ted said just went alpha this afternoon."

Also, this conversation is very different if you are an early stage company racing to market as opposed to an established organization where P&L often takes precedence over R&D.

Because so much had been invested (sunk cost fallacy), no-one could really get their heads around a shift to SES, even though it would have been a slam dunk in improved reliability and developer productivity.

Whereas if we were on, say, Mailgun, and then someone wanted a shift to SES, that debate could probably have been a more rational one.

I just point this out to say that investing in your own infrastructure can be a very real form of lock-in itself.

For small companies, you may not be able to afford infrastructure people, and moving fast makes way more sense. There's little point in paying for an ops person when you have very little infrastructure.

At a certain scale though, AWS stops being cost effective. You begin to have room in your budget for ops people, you get room to afford datacenter costs, and you can start paying for a cloud architect to fill out internal or hybrid cloud offerings using openshift or openstack.

It's all about the right tool for the right job.

I know for a fact GE is moving a lot of their workload to AWS.

Yeah, Netflix's opinion is to use Amazon as little as possible. Their critical infrastructure (the CDN) is not anywhere near the slimy grip of AWS.

This is simply untrue. Everything but their CDN uses AWS.

>Their critical infrastructure (the CDN) is not anywhere near the slimy grip of AWS.

The streaming website and app aren't critical infrastructure? Databases containing all of their business and customer details aren't critical infrastructure? Encoding content so it can be delivered by the CDN isn't critical infrastructure?

That's like saying I don't trust Ford because I buy Michelin tires while I drive a Fusion.

Netflix does presentations every year at ReInvent about how they use AWS and they have a ton of open source tooling they wrote specifically tied to AWS.

This ease of use philospohy goes way back to the one-click patent. If I want DNS, why wouldn't I go to amazon, which has all my finance details, and a decent interface (and even API), rather than choosing another DNS provider, setting up my credit card, and having to maintain an account with them. So I choose DNS via AWS. Then I want a VPS, but why go to linode and have the overhead of another account when I could do lightsail instead?

If you use provider agnostic solutions, things get expensive quick.

Stuff like SNS and Cognito is much cheaper in terms of TCO.

If you don't use them you can switch providers more easily, but if you use them you wouldn't need to switch providers.

On top of that. I lose the “easy button” of depending on our AWS business support contract if something gets wonky.

Can he do all that across multiple geographically dispersed redundant data centers?

Having relatively easy to spin up alternatives is a great thing. I can run my application entirely on a local kubernetes cluster or one on Amazon, DigitalOcean or Google's cloud services. That sort of flexibility is excellent and has allowed us to scale into situations where we otherwise couldn't have affordably done so (being able to buy some bandwidth from Joe Entrepreneur has it's benefits sometimes).

Which compliance regulation require you not to use a cloud provider? At most they may require you to not share a server with another company - that can be done with a drop down - or the data has to be hosted locally - again that can be done by selecting a region.

The policies that say not to use a company controlled by the US government. Or the ones that say under no circumstances should the data be sent over the Internet to a third party "because OPs are hard".

Even if you chose to use AWS’s OLAP database,Redshift, it uses standard Postgres drivers to connect. You could move it to a standard Postgres installation. You wouldn’t get the performance characteristics of course.

If you don’t want to be “locked in” to DynamoDB, there is always the just announced Mongo compatible DocumentDB. Of course ADFS is used everywhere.

Why in the world would I want to manage a colo with all of those services myself and still not have any geographic redundancy - let alone any place near our outsourced developers?

sorry -

isn't the _entire context_ of this article (and your response, experience, etc) 'implications of changing an underlying infrastructure' ?

yes, this isn't something one does on a whim, but it does happen, as your own post suggests.

Yeah, nowadays it's done by dissolving the company or being bought out, then shut down.

Still, it's the same thing. Use of inadequate infrastructure being, let's say, terminated.

What do you consider the lock in when using RDS? S3? EC2?

If I did want to move from AWS that’s the first thing I would do. Put an API end point in front of my business logic, change the event from triggering lambda to an SNS message and move my API off of AWS. Then slowly migrate everything else.

People aren’t using the same frameworks and infrastructure they used 10 years ago.

The cool kids don’t do large apps, we do microservices. (Said ironically I’m in my mid 40s - far from a kid.)