That's my experience as well. Generalists are often the most productive people. A good generalist can often grasp the whole system faster than a niche programming language specialist. And you usually don't need those niche programming skills for 99.9% of problems. Humans are capable of much more than the small niches companies put us in.
Generalists are generally (hrhr) ignored by HR though. They don't fit any mold so have a harder time getting hired. The most successful ones I know usually freelance.
Mandatory plug: if you think you'd fit the bill and are keen to explore, find me through my HN profile. Bonus points for being anywhere in APAC, although I can refer you elsewhere too.
I have the title right now but it is a new one in the company and it is up to me to navigate really what it means for us.
EDIT: And are there any books or other resources you or you see Solution Architects recommend?
I think as you gain experience you naturally become more of a generalist over time.
I've been a freelance developer for 20ish years and I would consider myself a generalist at this point, but it took many many years to get to this point.
Everything from learning the Linux basics, how networking works, various programming languages and web frameworks, database modeling, dealing with the ops side of things, and the list goes on and on. Basically I spent a majority of my life learning all of these things in a lot of detail, but most of it ended up being on a need to know basis.
Basically once you become somewhat proficient in something, you naturally start learning about some other related topic. Even today I still think I know almost nothing in the grand scheme of things.
PL specialists are also pretty good generalists. I mean, what hasn’t Jeff Dean done and do people even remember that he has a PL PhD anymore?
PL specialists are also pretty
good generalists.
Jeff Dean, Fergus Henderson, Ken Thompson, Rob Pike, Ben Gomes, David Stoutamire, David Bailey, Urs Hoelzle, Hal Abelson, Martin Abadi, Peter Norvig, Sanjay Ghemawat.
There are more but can't think of them off the top of my head. Chris Lattner is a non-Google example.
(As an aside, I see a similar phenomenon -- albeit at a much lower level of ability -- in my undergratuate students, and I've supervised a large number: top students almost always do well in / and have enjoyed their compilers course.) Why would that be the case? I can offer 3 hypotheses for discussion.
1. PL / compiler stuff is so trivial that everybody can be good at it.
2. PL / compiler stuff is so difficult that only the smartest can be good at it.
3. PL / compiler stuff forces you to think deeply about the most important tools in programming, namely the CPU and programming languages, and the bridge between them (the compiler).
I'm inclined to go for (3).
I don't think it adds up to generalist though. Generalists need knowledge in a lot of different things (though one man's generalist is another man's "full stack" web app developer), or else they're just smart people who don't know much yet.
This is to say that I don't think you should use Jeff Dean as an argument for hiring PL specialists in generalist roles.
But I work with people who are extreme specialists. (I don't mean in React. I mean in computer vision or non linear optimization, that kind of thing.) There is an entire class of problems that only those people can address properly; I could learn to but it would take me a couple years and the output from these years would be significantly below state of the art. In general they're skilled, not just talented, but they also work with mental models that require some amount of essential ability to grasp and that plenty of people will never be able to fully understand.
Although, the most successful developers I know (of) are both generalists and extremely talented. Their special talent is "people understanding"!
I loved the work, but the downside was no one ever advertised for that role despite it was very important to the project.
That doesn't mean much, as in the not so distant past (and in some small shops even today) the resident database expert is far from experienced or even technically apt. In fact, it was and still is a job that was assigned internally to someone who happened to have anything to do with the project, who was thus forced to accumulate yet another responsibility.
Older programmers: tired of having to be "flexible" for decades; extremely deep knowledge, siloed
Most clever programmer I saw was a guy in his 40's (in early 2000). He was a layout engineer who wrote a PowerPoint+VBA design rule checker for extremely rapid prototyping of layout ideas.
Most clever team I met was a group of 30 somethings (mid-1990s) who wrote a CPU architectural simulator that was essentially a tweak of the first linux kernel.
Most "how the fuck did you do that" was a 25 year old out of Russia who wrote a Windows 10 hack for hooking system calls for performance monitoring. He did things Windows docs said you couldn't do. All un-commented assembly. Glad I left before I had to own it!
I’ve had that surreal “we really want you to stay but we can’t give you a raise” conversation too many times in my career.
Then a subset of the gifted technical individuals grow tired of management, step down and go back to the tech to do what they really loved all along.
I've met a fair amount of brilliant engineers in their 40s/50s who have been through management and decided it's not for them and are now shining as engineers again after a decade lost to leadership roles
To answer your question, they get swallowed. Google etc hoard talent, and those that can fake it. I don't know if that's good, but they definitely end up with resources to produce incredible things. Then again what would they do alone, I wonder.
Learning is the key to flying high when it comes to a knowledge intensive career like programming.
Its hard to do that a while. Also early 20's you just exit teenage, and a have degree of naivety and foolishness. That spark goes with time. And you become more intelligent, calculative and serious to stop experimenting the way you are before.
But mostly, they stop learning. That's what causes the downfall.
Of course it's for different reasons. Something many people don't want to admit is that a lot of talent comes from practice and experience, and someone in their 40s is much more likely to have 20+ years of experience than someone in their 20s.
The early-20s geniuses I know are talented for different reasons. The ones I am thinking about are people who pick up new things absurdly quickly; pick up on patterns very easily; have general experience in enough domains to make quick connections; etc. But some of them are extremely undisciplined (which may or may not be related).
Those same people with age, gain not only experience but also hopefully discipline. A combo like that yields you people like John Carmack and the like: Not just talented, but out-of-this-world "next level".