I think some problems are just seen as sexier than others.
I just wanted to shed a light on some unfortunate external pressures faced by a subset of engineers, which contributes to this problem. Didn't meant to come across as cynical.
My solution was to work someplace that can’t afford to waste money on bullshit. Has worked well so far.
It's a great concept, and is often what irritates people. I work at a national lab, and senior management is absolutely obsessed with metrics, often to the detriment of the purpose of the organization itself.
There needs to be some humility too - especially in our modern data driven world. All that is important, you may not be able to measure, and all that is measurable may not be important.
Certainly not in Amazon. Surely there can be exceptions but in general the company has a culture of simplifying stuff anywhere is possible.
IMHO the largest contributor to over engineering in this company is people suggesting flaws in other's designs simply to have something to contribute during a meeting. I can't remember anyone, ever, telling me to remove something from a design doc (3.5 years).
I have added unnecessary complexity to my own designs as a response to comments. Not customer driven, not data driven, but somebody at a meeting got focused on something and it ended up getting added to the design.
I've had unnecessary complexity introduced to my code during code reviews as a response to comments, too. Most often it's to make things "more testable" so as to reach an arbitrary code coverage target.
https://raw.githubusercontent.com/aws-samples/aws-refarch-wo...
Ironically, that's how you are supposed to run Wordpress on AWS.
My original comment was 100% based on my experience as a developer at Amazon.
Most of the time it's people from the team that is hiring and one or two "guests" from other teams (but usually working in the same building).
Managers also have plenty of power to influence the decision, therefore keeping a very uniform hiring bar is really difficult. (But no, it's not "completely arbitrary")
Also, I wrote that the bar dropped a lot, to the point of shifting all employees level up by one, but this does not mean that the company hires 90% of the candidates.
If a team was hiring 1 candidate every 1000 screened resumes and now it's 1 in 100 it's a whopping 10x change... but that doesn't make you a terrible engineer!
Say one slightly dubious thing and you torpedo your chances of the job. Write one slightly dubious thing on your CV, and it will be rejected without a second look.
While I’m confident that most of the people rejected for those reasons would be at least adequate at their job.
>They're afraid.
In many companies (think FAANG), engineers, especially senior engineers are incentivized/forced to show fancy design docs as part of the annual appraisal process. The more complicated the design, the more 'foresight', the better.
If it sounds kind of ridiculous (TPS reports from Office Space anyone?) it is. But on the other hand, taking a bit less cynical look, the more massive a company gets, the more the voices which demand objectivity in all these promotion/bonus multiplier processes. So a kind of obsession about such weird 'measurable' metrics gradually builds up in the name of objectivity.
And as soon as there are metrics, you can bet everyone in the system will do their best to game them (it only makes too much sense to do so).
And thus you end up with over engineered systems all over the place.
A lot of the 'not-invented-here, let's reinvent it' style culture also develops similarly - you have too many smart people in a room where the work is just not that demanding. Even if you were to get over your personal existential crisis (why am I writing yet another crud app?!), if you're the type that wants to see a promotion every other year, your're forced to invent work this way.