Even the best developers get ignored if they try to justify pure-tech-debt fixes. So they learn to include fixing tech-debt as part of a fixing a problem that has some _direct_ business relevance if addressed. This gets clearly observed and taught to the all tiers of developers further obscuring the rationale for architectural changes from more senior management.
"for funsies" probably isn't that far off. Because the process is more like someone gets interested in something at some point. Then at that point +~1-6 months someone raises a problem and some senior dev gets stuck on the idea that the awesome thing they read about can solve it. Then before you know it whatever tool they want to use has more bells and whistles than the average mars lander and does everything short of curing cancer.
There's rarely any good correlation between the problem and the solution. That gap can just be bridged by buzzwords. The true correlation is usually between the solution and whatever the most senior dev on the team thinks is shiniest at the moment.
It's not always in one's control to avoid complexity. The simplest solution to a problem in a lot of cases may be 2-3x the lift (simplicity tends to require more work, complexity is easy) and thus blocked by the business. A holistically simpler solution may be blocked politically because a certain team doesn't want to own some functionality etc ...
I would say the best developers can see complexity coming and have a healthy fear of it, the medium devs don't mind complexity and the worst devs can't get anything done without increasing complexity.
This is such an important point. For whatever reason it has become ingrained in people's heads that the simplest solution must by reductionist logic be the easiest one. And therefore the easiest solution is the simplest one and it is good to be lazy and just introduce complexity everywhere.
This is where the whole "MVP" concept got out of hand. MVP didn't mean an overly-simplified prototype. It meant solve one, narrower, problem well. This also pairs well with PGs "do things that don't scale" advice. You are taking on what others might think of additional complexity, to solve a very targeted problem more effectively from others, because using either deeper analysis or first-principles or whatever, you've actually better modeled the underlying complexity. Then you try to scale given those insights.
You see this a lot with writing as well. It's very easy to ramble on, it's very hard to concisely convey your point.
That's much harder than doing less simple work on your own team
I've seen no evidence that companies are even trying to hire developers who "avoid complexity". If anything, the interview processes are designed to select for engineers who bathe in complexity. There are so many interviews which consist of "How would you rewrite from scratch this thing that already exists?"
There is such a shortage of people that know the basics of programming, that selecting for such l33t skills is out of the question.
The hiring process right now is not about selecting the best, it's about selecting those that pass some low bar.
On the other hand, great for us who can code :D.
So if that isn't part of the deal, I don't even bother.
"How high do you score your C++ skills out of 10?" "9/10" "OK, can you explain what a pointer is?" blank stare... "no idea"
I would definitely consider pointers pretty fundamental if you're a C++ dev.
It is pretty hard to believe that 50% of candidates don't know what a pointer is. I've barely touched C/C++ and still know what pointers are and how they work.
Congratulations you’ve successfully triggered the pedantic interviewer. You will now face six questions on pointers in C++ they just looked up each more trivia based than the last.
I'm one of those, and it really gets on my nerves when systems are overengineerd, or use tech that has more drawbacks than benefits for our specific case.
This is always a challenge in general - how do people learn the lessons of complexity without creating it and then seeing the effects? I wish there was a better word for it as every person who reads "complexity" says well "duh of course I don't want that" before they then go and manufacture another bucket full of it. Complexity masquerades as simplicity - in the first instance it nearly always solves the immediate problem better than anything else. Recognising the latent complexity of choices is one of the hardest but most important skills to learn.
I think there is a tension between this kind of actively guiding anti-complexity management and hiring "top talent".
The very best developers are capable, and avoid complexity. The next best developers are capable, and love complexity. The worst developers are not capable.
There aren't enough of the very best developers for a company to plan around hiring only those. So, if you want to hire developers who are at least capable, you have to give them some leeway to make things overcomplicated. Yes, that incurs a real cost to the business. You can think of that as just being another part of their compensation package.