This makes a lot of sense. The same thing happened in the past with new technology, such as the electronic spreadsheet:
"since 1980, right around the time the electronic spreadsheet came out, 400,000 bookkeeping and accounting clerk jobs have gone away. But 600,000 accounting jobs have been added."
Episode 606: Spreadsheets!, May 17, 2017, Planet Money
Given that the US has transitioned out of manufacturing and into businesses services - I don't think much of this is explained by technology creating new jobs.
I think it's just explained by the workforce growing - and the shift in the US's role in the global economy.
So the real question is - whether the ratio of different roles changed vs total population.
"Though the spreadsheet was supposed to make clerks obsolete, it in fact just upgraded their job requirements to allow the operation of spreadsheets"
And probably their salaries too.
This is more in line with the discussion related to OP, as well, as devops employees exist, even though apparently it is automated.
Go back and read a few DevOps books and blogs by the founders of it. We will always need separate disciplines for dev and ops, just like we need mechanical engineers and mechanics/racecar drivers. But we need them to work together and communicate better to solve problems better and not throw dead cats over walls.
You can of course give devs more powerful tools, more access and agency to enable them to develop the software better. Sentry.io is a great example of what is needed; makes everyone's life easier, devs can diagnose issues and fix bugs quickly without anyone in their way. That doesn't require operations work because it's just simplifying and speeding up the triage and debug and fix and test phases. It's the fundamental point of DevOps.
You want the person who designs the car to know what a car is, and to be able to diagnose basic issues like "the fuel gauge says 'empty' and engine won't start". And there's no analogy to an Indy car driver in software, every distributed system is self-driving.
The most popular alternative to "DevOps" is a team of developers who do not run the software, and may not even have the skills or capabilities needed to boot up the server process. They do their development in an IDE, run unit tests to verify functionality, and do not have permission to log in to the production environment.
Meanwhile the "ops" side consists of people who may know basic shell scripting, or some Python if they're a go-getter, but are unable to read stack traces or diagnose performance excursions.
---
> So you're familiar with Six Sigma then? Value stream mapping? TPS?
> W.E. Deming? Martin Fowler? There's more to DevOps than deployments
> and CI/CD.
> I haven't worked at Google, but I expect somebody gave you some tools
> and some access to cloud infra and said "good luck".
> If you're lucky there was a small army of ops people behind the
> scenes keeping things running for you
> if you weren't lucky you were expected to know how to correctly build
> and run complex systems at scale by yourself
> > every distributed system is self-driving
>
> I didn't know Google was in the bridge selling business.
As an outsider, that command looks really easy to mess up.
* shell interactions with quotes
* double quotes
* interpolating into image namespace with no autocomplete
* easy to forget an argument
* do you get autocomplete against the deployment name?
* var x = new List<Dictionary<CustomType,int?>>()
You get:
* go to definition of `CustomType`
* autocomplete on classnames
* highlighting on type mismatches
* warnings about unused vars.
* if you initialize the var, the IDE will try hard to not let you do it wrong
1) in the code realm, for doing minor work we have really strong guarantees,
2) in deployment realm, even though it's likely going to hit customers harder, the guarantees are weaker.
I think this is behind the feeling that the k8s system is not ready yet.
>> None of those have any relationship to DevOps.
Deming. I think you mean maybe that Deming has less of a relationship to SRE (Google style). The DevOps Cafe guy wears a Deming t-shirt and talks about Deming all the time. Maybe he has denounced Deming for some reason?
Value stream mapping. This has been brought up in at least one talk at DevOps Days.
Six Sigma. There's some obvious overlaps if you look it up.
Cool, you probably know my old colleague Hugo, he was one of the first SRE's at Google in a team of like 20 or so. (I wasn't there, could be mistaken).
Anyway, what "Production" was, is different than what devops is. DevOps is different things to different people. In the beginning it was "What happens when systems administrators do Agile".
(Genuinely), Confusion came about because of the "100 deploys a day" talk and the conference being called "dev ops days" (to include developers).
The two things got conflated and now everyone thinks devops are.. well, build engineers or sysadmins or developers who learned a bit of bash and terraform.
From what I have seen, it's also getting familiar with the inner workings and process of the company, as well as current and past runtime tendencies for performance and uptime.
That sounds like sysadmins.
DevOps is how Engineering respects the work of sysadmins.
You typically build systems to sell a product. The driver is the customer that uses the app. You build the app to be as simple as possible to effectively operate while providing the best possible outcome for that customer. The driver doesn't care or need to know how the engine or the rest of the car works, just that the engine and car works well.
- each role in a company tries to optimize/nudge whole organization toward this role's convenience.
- specialization improves local optimum (advances certain role) at the cost of global optimum (everybody has to dance around new roles processes)
- joining seceral roles into one, creates the oposite result, optimum is searched at more global level (not necessarily found)
- Separation of responsibilities (aka creation of new role) can generate a fractal (e.g. tester of left winglets blue stipe's thickness meter)
- complete joining off roles will create homogenous chaos after n employees (everybody should do everything)
prediction: we will see constant experimentation, first roles will be split, then some will get joined. then split again. then joined again. (people cant search for local optimum and global optimum at the same time)
Lesson learned: kubernetes isn't that bad
I agree with this definition of DevOps. However the vast, vast, vaaaast majority of real life uses of the term "DevOps" I've seen are just rebranded sysadmins. Sometimes it at least implies a more engineering approach to their coding. But in these institutions the Devs and Ops are very much separate groups of people, unfortunately.
If you're lucky the hiring manager might have at least read the Phoenix Project or the Google SRE books.
SRE is a specialized software engineering role -- you'd hire SREs if you wanted to create something like Kubernetes in-house, or do extensive customization of an existing solution. If you hire an SRE to do sysadmin work, they'll be bored and you'll be drastically overpaying.
DevOps is the idea that there shouldn't be separate "dev" and "ops" organizations, but instead that operational load of running in-house software should be borne primarily by the developers of that software. DevOps can be considered in the same category as Scrum or Agile, a way of organizing the distribution and prioritization of tasks between members of an engineering org.
---
With this in mind, the question could be reframed as: if projects such as Kubernetes are changing the nature of sysadmin work, why has that caused more sysadmin jobs to exist?
I think a general answer is that it's reduced the cost associated with running distributed software, so there are more niches where hiring someone to babysit a few hundred VMs is profitable compared to a team of mainframe operators.