Sure, I just created a CodeStar project which I guess hooked me into 5 or more of AWS services but now what? What if I want to create a DynamoDB or move off of CodeCommit? How do I know if it scales or is redundant? How much does this beast I just created in a few clicks cost / month - can it be cheaper? etc. etc.

It may make DevOps jobs easier or help people get into DevOps but it certainly doesn't replace a trained AWS architect.

But on my last project (much bigger) Elastic Beanstalk started to really show its rough edges, and I was pretty close to tearing it down and setting up by hand, which gets in to becoming the guy who knows AWS Good Enough™.

But I could've easily seen that project scaling way up and being tempted to go the Kubernetes route, which would've required a true DevOps person to maintain.

Just based on my anecdotal experience, I don't think either of the previous projects were paying for the DevOps person in the first place, and I don't think CodeStar is going to replace them now either. This is just low-hanging fruit.

It was a node.js project and the db was separate, and keep in mind my knowledge of AWS is/was limited. But then, this seems to be the target audience for EB, yes? I'm sure a lot of this could be fixed but...

1) Logging

It somehow took me significant time to figure out how I could send meaningful logs anywhere but poorly organized flat text files in S3 without adding some kind of logging sink to the code itself. Sure, I could've just plugged in Winston and spat out everything to MongoDB but the team couldn't understand why we couldn't just get logs out of EB. And yeah, we could could grep through the S3 bucket but that meant syncing the whole bucket and trawling through an absolute ton of log files with no more precision than your knowledge of grep and regex. Not the best thing.

I eventually dumped all the logs in to CloudWatch, but the documentation to do so was severely lacking (from my perspective) that dealing with CloudFormation template files made me consider ditching EB entirely -- If I'm dealing with this garbage anyway why don't I just go whole hog and get more control anyway?

Even after dumping the logs to CloudWatch, I was still "the guy" for doing any good searching of these logs. Maybe this was just a team quirk, but what they really wanted was logs in MongoDB or something they know, rather than something AWS specific. Again, a logging sink would've fixed this, but there were concerns about uncaught exceptions and attempting to send to MongoDB in an unknown state.

2) Deployment Time & "Health Checks"

The team was used to pulling the latest changes manually and restarting the node.js process. Introducing CI ironically increased friction as our deploys were taking an upwards of 10 minutes (20 if immutable). This doesn't sound like much of a problem in a traditional environment, but going from a "cowboy coder" environment to "why the hell does it take 20 minutes to deploy!?" was a stretch. My initial answer was "health checks!" but then we discovered that the health checks weren't actually doing much for us... I had erroneously assumed that part of the health check meant monitoring the responses of the web service and aborting deployment upon massive failure. We had a deploy that crashed and returned 500 on every request, yet somehow didn't abort. Part of this may have been because the developer involved in the bug did everything possible to prevent a "crash", instead of letting the process die upon unrecoverable error (part of a frontend JavaScrtip mindset I assume), but trying to sort out how to make the health check look for 100% 500's brought me even further in to "why not set it up myself?"

3) Deployment Failure

We hit a hard limit of deployments and had to clean out a list of artifacts we didn't even know existed before being allowed to deploy again. WTF? This is when the team started to really question the value EB was providing.

4) I ended up "the guy" anyway, so what does it matter?

The documentation was too dense for my team, and the quirks were plentiful enough that I ended up "the guy" anyway. One day I came in late and a deployment had gone wrong. EB had been rolling out bad code for 20 minutes and they didn't know how to fix it. I had to clean up the mess. That was really the "fuckit" moment for me. Hand managed EC2 instances wouldn't have caused this problem. Regardless of knowledge problems within the team itself, the whole point of adopting EB was to avoid needing infrastructure specific knowledge to scale & deploy. Now EB was doing more harm than good.

---

These are just the things I remember off-hand. I'm sure there's solutions to these problems, but it just wasn't worth it for me to investigate when really my purpose was just to write and ship code, not futz with the infrastructure. We would've run in to less problems with a naive, manual setup in the end.

You can ping me at rtannerf dot com if you want to know more. I'm sure I can dredge up a few more annoyances and clarification from an old coworker. Hope that helps.

More and more work can either be outsourced to services like AWS is building or cheaper IT professionals anywhere in the world. The only question here is when taking the lead outweighs the costs.

If this statement is not true, then Amazon would say that AWS is failing to fulfill its mission.

I could definitely see less ops engineers needed in some companies that move to AWS from some in-house platform but overall "devops" is rapidly increasing in demand.

Bezos is maniacal about winning, and I'm sure this will win where they want it to win.

Much of my value as a consultant is that I can drop into a shop, understand the tooling and the stack, and develop a solution that makes sense, using AWS tools where appropriate or general-purpose ones where the AWS ones don't fly, and I can make it approachable for developers (not operators, but developers) with less experience and make it easy for them to hammer on it and expand it as needed.

That's not going away--if anything, the steadfast drive towards understanding less of how anything you do works makes people who do understand it more valuable when the shit hits the fan. And it always does.

I should mention it's a very small team and the option to split the DevOps work does not exist. I'm still evaluating GCE and friends, because the amount we pay for Heroku is ridiculous.

Edit: Also I feel like a need an AWS <-> English dictionary

The set of shops that actually benefit from k8s or the like are a rounding error and so many person-hours are burned building stuff like this that it hurts. Simplicity is better than complexity until your needs require complexity.

If you'd like to chat further, feel free to reach out - email's in my profile.

I'd be interested to hear about experiences using CodeCommit, CodeDeploy, etc. but I doubt AWS found the One True Workflow.

I admit, it wasn't the easiest thing to setup. But its nice. When I commit my changes to my mainline, it deploys the code to the host and configures everything for me, and keeps all my hosts up to date.

Coworker uses it all to kick off lambda functions.

CodeCommit is just a basic Git setup, so nothing special. But working with CodeDeploy and Pipelines is ok. I wouldn't say its much better or worst than other products, and you can use them to deploy to non-aws environments also.

I don't know, I can't really sell it well. It does what it does, and it works well enough for me.

Is this because the further they move up the stack, the more UX matters? That's Amazon's perennial weakness.

Remember that Bezos says, "Your margins are our opportunity." They will get around to gobbling your market, unless it requires something that doesn't scale for them.

AWS's strategy is not looking for the one true workflow, its to envelop enough common usecases that it dominates nearly the whole field, giving you as custom a service as you want.

I call it the "Atlassian strategy": be mediocre at enough things that you become enormously successful.

It's a long way from a death knell to competitors though.

AWS needs these to prevent other players encroach their ecosystem. The higher the upper-stack offerings in the tech stack, the lower the quality is requires to be dominant. But the ecosystem will secure the customers.

It's not that you cannot beat AWS in one or two offerings, but you cannot beat them as an ecosystem.

Very true! If you only offer one piece, then you have to

1. integrate with them and other providers

And

2. Be substantially better (2x, 5x?) in order to beat the default provided by AWS

And

3. Don't forget you are competing with open source options as well (where the cost is the time to set up a solution, but no cash outlay)

As is, I wouldn't use it without an extreme organizational requirement.

Naah...It probably will be for more than 10 years just good enough for prototyping your applications.

Ops engineers have two choices: adopt a symbiotic relationship with the tooling, where they can be even more productive than before while wielding the tools, or eventually lose their jobs.

edit: typo

CodeStar (and its competitors) doesn't eliminate the future prospects of ops engineers. Instead, they reduce the grunge necessary to get something working. (After all, how much fun is it to actually set up and configure Jenkins ... for the hundredth time.)

So it's possible that over time, ops engineers get more time to worry about things that are harder (or impossible) to automate. For example, organizations run similar, but not exactly the same dev workflows. So understanding your existing dev workflow, and how to tune it is an ongoing thing. And with something like microservices, maybe you have dev workflows that vary by team. Or how do you tune your monitoring infrastructure to let you root cause faster, instead of just taking stuff out of the box. Etc.

I could be wrong, of course. Curious what others think.

You just need to shift from "I know to edit this /etc config file" to "I know to build this on aws|gc|azure". It's actually quite liberating.

Pivotal Tracker for life!

Really, if you're not okay with your job description evolving substantially every 5-10 years, then a career IT is going to be a source of a lot of stress for you.

I love this stuff. I deeply enjoy watching technology develop and become more efficient. This feeling is something akin to watching your kid drive off to college: I'm probably not needed as much anymore, but I'm pretty excited to see how far it has all come. That doesn't mean there isn't a selfish pang of nostalgia for how things once were (and a little bit of existential fear for my future financial wellbeing).

> it's starting to look like I may not be needed on the bleeding edge at all.

If it's any consolation I'm pretty sure this has been a theme of worry for hundreds of years, if not since the beginning of technology. Thankfully humans are extraordinarily adaptable (and bad at future prediction) so a combination of evolving our skills and failed promises means there's probably going to be a lot of work for us to do for a very long time.

> I'm probably not needed as much anymore.

I think this too is just part of successfully doing your job. And getting old. Hopefully you get to move on to new things as you hand off or automate the old. And then someday you realize that the twenty/thirty-somethings are really sharp and maybe woodworking ain't so bad after all. :-p

From first pass, saying CodeStar is going to kill DevOps is like saying that Lightsale (the VPS thing) is going to kill DevOps. At the end of the day it gets you in the door with AWS until your company can be adult enough to build something for real.

If anything, I think its good for DevOps. CodeStar could mean less hacked together junk at the get go. So when the real DevOps gets hired, less to cleanup and unf'k.

But DevOps ? wasn't/ins't it always just a matter of time, for most small/medium sites, until someone hides all of the complexity behind a nice, smart interface and leave very little for people to do?

The nice thing is the technology moves. In fact I imagine that's what attracted you to the industry in the first place. As these things roll forward we start doing more and the pie just grows.

Maybe the future of DevOps is in managing, deploying and optimizing AI stacks? Or maybe just managing the -cost- of AI? Or how do we manage and exchange personal health data on these platforms? Or biotech experiments? Or global IoT stacks? Maybe it's managing APIs, or solutions that are federations of third-party services. So much stuff in the technology pipeline it blows my mind.

Some of the harder skills may become less relevant (they'll never go away, in actuality). The principles and soft skills you have still have a long way to go.

This stuff is not as easy as vendors or aws would have you believe.

Anyone here have experience transitioning from being ops to a behind-the-scenes cloud engineer?

I imagine that market is plump full of CCIEs, etc.

mainframe, timeshare, client server, cloud, it goes around and around