Every year since my first year, I wrote on my self-evaluation form "I really need to learn more about (GPU computing/differential geometry/hyperparameter optimization)". And every year, I got told "be nicer to people, show your work, thoroughly explain your algorithms, double-check". (Well, different things every year, but all in that list.)
The fact is, intellectually challenging work doesn't exempt people from, well, doing work. Keeping records, using processes, talking to people. There's no market for heroes who only want to pull the sword from the rock but not get any blood on their shirt.
I think the key fact of my landing-the-plane story was that there was a hell of a lot of technical knowledge used in landing the plane. They just didn't do barrel rolls on the way in.
It's not the lack of technical skill. The broader and deeper your range of skills, the generally more-useful you can be to people. It's putting the problem first instead of your own (perhaps) boredom.
Putting the problem first is what's creating all those "soft skills" books the OP mentions. When we put the problem first, suddenly we realize that we're bad at code standards, or negotiation, or interviewing, or any of a dozen or two other skills that we may never have wanted to learn.
That sucks, but that's life. That's the job for most of us. When you truly put the problem first, it dictates to you what skills or knowledge you might need. You are no longer taking an active role, choosing from a big closet of cool-looking weapons to go do battle. Instead, you're taking a reactionary role, carefully looking at both the people-systems and the technical-systems to feel your way through an engagement that will deliver the maximum value for the least amount of work, ie, putting your client's interests ahead of yours.
We don't teach people to think that way, so you'll see a lot of this soft skill stuff out in the industry that's an effort to correct the training they should have gotten in school.
But yes, absolutely, pick up more CS skills. I love that stuff. Starting in on Category Theory myself next year, probably Haskell too. I plan on having a blast.
But my fun or enthusiasm is not at all important to consider when I'm trying to help folks. Then I put their needs ahead of my own.
I'd also add that it's not just soft skills and deeper CS skills. A good liberal arts education pays off every day. I think tech folks look at these books and think there's nothing there -- then go off and make the same freaking mistakes over and over again. For those folks, trust me, there's something there. You missed it.
The disparagement aimed at the liberal arts these days (at least in American education) is really quite saddening. The prevailing notion seems to be that the purpose of an education is to produce good STEMs, not good human beings.
I'd just like to add that the world is full of fascinating (yes, even tech/math/cs heavy) problems. If you find your problem (as distinct from the tech stack or whatever) boring, you could always find a different, cutting edge, problem to work on the next few years (or decades)
You might have to get a PhD/demonstrate extreme dev skills/build cutting edge software etc to get paid to work on such problems, but them's the breaks.
You don't have to 'settle' for boring domains/problems if you don't want to, but you do have to put yourself in a position to get paid to work on them, and some "soft skills" might come in handy (though I doubt the value of some books in that list. 'clean code'? bleh) .
They were just getting started -- yet there was this ton of technical frameworks they just knew they needed.
For the ones I knew, I went ahead and applied, of course. Even got a few of them. But it always amused me that they could be so sure of all the frameworks and tools -- and not have a clue as far as to what the code was going to do.
This is just a nerd version of "we've formed a club and are looking for other cool kids to join"
Fine enough, I guess. Sure were a lot of those projects that failed, though.
These are clearly hard skills for the individuals that complain the most, or suggest they aren't important. It's much more comforting to work on technical skills which you believe can be measured objectively, but at some point you need to focus on the human side which is messier.
I attempt to be more specific with resources I point people to if they ask for help, or I need to steer someone in a particular direction. Just telling people to read a soft skills book can be chore and they won't necessarily identify the right lessons to learn. People also need to understand that you aren't trying to turn them into a manager, or one of the people they don't admire. It sometimes help to point out that the role models obvious technical skills aren't the ones that got them to that point.
% of projects that failed due to some "soft skill" issues such as covered in these books: 100%
% of projects that failed because of some bit of CS somebody was missing: 0%
My enthusiasm is not curbed.
I think the funniest part of this comment is the fact that it's this exact attitude that makes people crash their projects: it's not tough enough. Surely there's some magic sauce we can add on top to make it tougher, right? Maybe the project is only keeping track of the 50k books and 10k patrons at the local library, but we'll certainly need a messaging system capable of huge amounts of realtime transactions, right? And what kind of cloud infrastructure is that going to take? Probably something complicated, I'm sure.
As a newly-minted instrument pilot, I was lucky enough to sit in the jump seat of a local commuter as it made its way home to my local airport. We were shooting an IMC approach down to minimums. I kept my mouth shut and paid attention to the two pilots working.
It was like watching a clock. Everything was dull, double-checked and cross-checked. When we finally landed, I told them "That was pretty boring"
I'll never forget the pilot's reply, "That's how it's supposed to be"
The vast majority of tech work is not intellectually-challenging. Even when it fails, it usually fails for these "soft" reasons.
Plenty of folks are not happy with this state of affairs. Those are the people you need to keep away from the keyboards.
The greatest skill you need to know as any kind of professional is the skill of dialing it back to only what's required by the client -- not what might be the most fun or interesting thing you'd like to do. That can be really, really tough.