Throughout my career, as tech became big(ger) business, I watched as management, marketing, and design, took away our keys. Scrum, as an example, and all that followed seemed to be a way for upper management to rein in engineering, give management "numbers" that they could show their bosses, etc.
Scrum is my least favorite part of my job, by far. I feel like I'm being treated as a child who can't accurately estimate project time, difficulty, or dependencies.
Yes. It used to be that tech companies built tech products and were run by engineers, that's what tech meant. It was glorious and I loved every second.
Now "tech" companies aren't really, they are actually in the advertising business or other forms of data gathering, movies, shopping, etc. All of which uses tech but isn't tech. So very boring.
And as you say then PMs took over from engineering and engineering has been reduced to a fast-food order cook with little to no agency, just bolt the right ingredients (or AWS components) together and give status reports every day. That's not creative engineering anymore.
Developers don't like hearing this but the vast majority of us _are_ effectively assembly line workers. We do little to no actual engineering, instead we plumb together pre-packaged framework, libraries, and software. We are treated as fungible because that sort of worker is intrinsically fungible; skill is involved in software development but not enough to elevate it from being a trade to being a profession. A symptom of this is the not infrequent HN posts complaining that "Nobody actually uses CS education in the real world"; assembly line workers don't need a degree to do their job either.
I started programming for Intel in the early 90s and loved being free from the menial labor, doing creative work, and being treated like an important, respected, and integral part of the company.
Fast forward 30 years, and programming feels like the same grunty ditch-work I was doing as a teen for the reasons you mentioned. I hate it.
What is wrong with that?
Dunno, when I see loose ends in specification, I report them to analysts with proposals how it should work to be useful for business logic. Well, maybe I am paid to think. What's the profit from doing a shitty job and then suffer from damage you inflicted on yourself?
Product companies: selling your software as a product means you're always going to feel like you're on an assembly line. Except you're not the assembly line worker, you're the assembly line.
Ecommerce and marketing companies: you build software that enables the company to sell some product. You're not the assembly line, you're not even working on the assembly line. You're designing the assembly line and building the machines. You're the engineer.
Whatever you do, always avoid companies that treat engineering like an IT cost center. The work culture is always dogshit and you're the first to go in a downturn. Look for somewhere that is "engineering first" or "developer driven" (e.g. somewhere a dev is high up on the org chart.)
Or to put it another way, if you're adding core value to the business proposition or enabling their core business you're probably not going to be seen as a cost center. If you're R&D or innovation focused, probably not a cost center.
For example, working on a Wordpress site for a newspaper, probably cost center unless they're extremely tech focused. NYT comes to mind, but they straddle the line between saas product and not. They're selling the subscription service which puts you in product camp but they also have engineering functions that work on enabling their writers produce content. The content is the thing being sold, not your software so its probably some mixed feels over there.
I should probably extend the original "ecommerce or marketing" to any more traditionally non tech business model that has built itself as a tech company. For example:
Working on a website for an architecture firm = cost center.
Working on cad plugins for an architecture firm to use internally = probably not.
And again, caveats, even for ecommerce you could be considered a cost center if the business is more like a traditional brick and mortar but they've "started selling online".
My personal hypothesis is that we’re increasingly being treated like assembly line workers. We’re not paid to think but to implement the poorly thought out hare brained schemes from the higher ups who have no clue about programming. This externalizes all the cost of poor quality software onto the developers (unmaintainable code, constant outages…) while reaping the short term benefits for themselves (promotions, raises).