Also I've noticed a lot of sub-fields like computer vision or self driving cars where there's a great deal of nerd interest but very little actual employment.
In fact, I have a feeling that computer programming is the stick that's holding the entire STEM tent up---if not for programming as a consolation prize you'd have a whole lot of physics and chemical engineers making lattes at starbucks---and further that web development is the stick that's holding the programming tent up.
I just used to say I am a computer vision expert, because in the ton of things I can do, this is a relevant domain.
Recently, advances in deep learning made half of the computer vision field obsolete. I just read some cutting edge DL papers, linked that to the things I know, and now I am a deep learning integrator.
I was taught in engineering school that my craft is learning and problems solving. I sell it as knowledge and expertise but the truth is that I am constantly learning in the domain I sell as my specialty, which is what everyone who is decent at anything actually does.
I'm strongly convinced that the less (nerd) sexy a field is, the more money there is in it for a good programmer.
The crappy programmer working conditions in the gaming industry is one example supporting this theory.
I swear, I repeat this comment in almost every news article posted about how "We need more STEM graduates" and "The STEM job market is super hot right now".
Yeah, programming is. I know so many graduates of a BS in math, physics, chemistry, biology, ecology that just couldn't find satisfactory employment. They either got a second, mostly unrelated degree (in management or something more industrial) or are working at a mostly unrelated job.
Even in traditional engineering disciplines, while it's easier to get at least OK employment the demand just doesn't seem that strong compared to CS. I have many friends that graduated with civil, mechanical, or chemical engineering degrees (in at least one case, an MS) who are vastly overqualified for their lab tech or drafter positions.
I have a strong background in mech engineering and robotics. Yet, for the two years I worked in that domain, I never had a single recruiter contact me out of the blue. Now, my mostly amateur computer skills somehow landed me a "tech" (read: IT) job. In the 2 months since I've changed my current title on linkedin, I've had 3 recruiters contact me.
I think this goes back to being an expert in a "domain". Being able to apply programming to that domain is far more valuable.
* I have a bachelors in electrical engineering from a top ranked school.
The future will be a new snazzy term. Same job, different title.
Software development is not engineering.
That is part of the reason for the high pay. There is always risk associated with specialization. You can't become an expert without specializing in something, but if you specialize in the wrong thing your effort may not be rewarded.
That might be the reason people demand higher pay, but it wouldn't be the reason why companies pay the higher pay. They pay it because there's a scarcity of people, and/or because they get some multipled value back. No company cares if I've specialized in the 'wrong thing' - they just won't hire me.
It's the classic example of complaining that there are no jobs, while the industry complains that there aren't enough talented people. I get a few recruiters pinging me weekly and I'm not looking to move jobs - their story is always that there is a huge demand for good people.
I think the CV job market is unlikely to get much better, at least in terms of numbers of positions available. On the other hand, it's a stable job market that isn't going anywhere. The pay is already above average and it's not likely to go down because of the experience required. There's no shortage of applications and it's not dependent on a particular language, toolset or hardware. Virtually every big factory in the world employs some kind of machine vision system, for instance. And pretty much every computer vision deployment has to be modified for the application.
There are much easier ways to make money though. If you want to be a wealthy programmer, get on the web/app development train and accept that you might need to pivot every couple of years.
It's a little ironic to hear talk like this, since around here I hear a lot of "sorry your coal mining job is going away, but you need to get with the times and train for something relevant". Same thing applies here. If you pick a programming specialization that eventually falls out of favor, buck up and learn a new one. Maybe that'll mean a pay cut for a bit as you ramp up, but that's just how it works.
It's a different story, I guess, if you go into a completely different type of programming from what you're used to (say Web apps to embedded programming), but I don't think that's necessarily easy.
Important difference.
For example, at one extreme, all tech lasts exactly three years; at one of several other extremes, 50% of tech is eternal and the rest lasts exactly 1.5 years.
My philosophy is that if PHP keeps the lights on, I better really know what it is, where it's going, what are it's limitations, because it would be pretty scary not to.
Plus once you learn a language or a database well, the next one is easier. The trick is knowing when it's time for the next one.
Don't mistake what I'm saying. I'm using the word "own" rhetorically. Obviously there will be no point in your career where you are likely to be the world's leading expert on computer vision. What you're aiming for is to be the leading expert that is available to employers within a reasonable amount of time at the level of compensation you're hoping for.
It's a spectrum.
Obviously, if you invest heavily in a discipline and it is decisively obsoleted, that's a setback. I'm just saying it's not an insurmountable setback, and I'll add here that it's still unlikely to leave you behind the pure generalists; in the process of building expertise and accomplishment in cryptography or OS kernels or whatever, you're likely to become a pretty far above-average software developer as well.
But the few people who are really hiring people who "own" CV want PHDs with a relevant thesis and you're not going to get one of those working weeknights after the kids go to bed.
Meanwhile people pursuing PHDs have no idea if their area of research is going to be hot in industry when they're done.
Granted, a lucky few people who follow the autodidactical route may find themselves authors of a really popular OSS project. Or they may be able to luck or maneuver themselves into a position at their current employer working in a capacity that future employers will see as "relevant industry experience." Obviously this is hardly a sure thing and if they plan it wrong or just get unlucky they're not going to have a nice outcome.
There are so, so many businesses out there that could benefit from CV, but they don't realize it. Maybe they're spending a ton of money on off the shelf OCR software that isn't great for their needs. Maybe they're spending a few hundred thousand dollars a year manually organizing and classifying documents when they could automate it.
These companies won't be hiring someone to own CV for them, because they don't know they need it. They might not even know what computer vision is. And they won't care about whether you have a PhD or a relevant thesis. They'll care about whether can can demonstrate that you can solve their problems and save them a pile on money.
Granted, finding companies with these types of problems is perhaps not something that a typical LOB programmer knows how to approach. And if they see themselves as just a LOB programmer, it might stay like this.
If they make an effort to get to know people and network outside of the usual tech circles, they might just find it possible to position themselves as solvers of specific types of problems - problems to which CV happens to be the best solution.
I know that's not a path that'll work for everyone. But it's one path that leads from 'LOB Programmer' to 'Someone who makes lots of money doing development with computer vision'.
As a counterexample, though, most serious work in cryptography (for instance) is done by grad students, but the best jobs in crypto are not automatically going to grad students.
Either way: if your goal is to maximize lifetime earnings, I think dedicating your career to "programming" is a bad call.
My point isn't that becoming a subject matter expert isn't a good way to supercharge your earnings for a few years. I agree with you there.
Clearly, a guy with lots of experience with and deep knowledge of the Oracle database could do pretty well for himself 10 years ago. A person with lots of experience with and deep knowledge of OpenStack could do the same five years ago.
My point is that it's not a good, safe or stable way to ride out the last half of a 30+ year long career.
My further point is that demand for lots of these specialists doesn't appear to be particularly elastic, and therefore "become a subject matter expert" is not a universally applicable bit of career advice for every LOB engineer worried about ageism: if everyone successfully took your advice there would be a whole bunch of unemployed crypto experts.
It's not my claim that simply by moving to a specialized subfield of programming that you'll instantly quadruple your salary and future-proof your career.
Obviously, you're going to start in some domain speciality as a commoditized practitioner and, with effort, move yourself to more elite ranks of that subfield.
My point is that it is better to work on that problem --- of being an elite in a specific domain --- than it is to be an elite at the general problem of building reliable software.
(It's "Burp", by the way.)
A guy with lots of experience and deep knowledge of databases in general, though, would do pretty well for himself both 10 years ago, and today. Even better if he worked on some database himself. Really, that's what owning a domain means: you don't want 10 years of experience as a _user_ of some technology, you want that experience as a _builder_ of said technology.
Are you sure about that? I mean sure, if we're talking someone who wants a deep learning research project to improve the metric for recognizing human faces from 99.7% to 99.78%, then we're talking about PHDs.
But there are plenty of "simpler" problems that one can work on, and plenty of more engineering-oriented problems that one can work on. Lots of companies just need to apply semi-standard techniques to problems that are specific to their business, and nothing more complicated or research-y.
(Being a PL design & implementation expert --- not "having very good taste in PLs"! --- seems like a defensible position in the market, except perhaps that there are a lot of people who want to work on compilers and you're competing with a healthy pipeline of them.)
In "flyover country" and the midsize college town I quite enjoy living in, indeed currently returns only a handful of BI-related postings for search terms like data analyst. Web developer returns tens of postings.
I'm not sure every programmer should start with trying to specialize in a domain. But definitely, at some point, you're reaching diminishing returns for getting better at programming.
Put another way: Getting better at the craft of programming allows you to write programs faster and better. Specializing in new domains allows you to do things you couldn't otherwise. This is the realization that made me decide to stop learning new programming languages, and instead spend my time learning new domains - I realized that learning Haskell, while incredibly fun and rewarding to my abilities as a programmer, will not let me make fundamentally different things than with Python. Learning e.g. Deep Learning, however, would.
Btw - even learning totally other skills - e.g. learning 3d modeling helps me a lot in gaining deeper knowledge of 3d than I would otherwise have - even if it's not 3d programming per se, it's still relevant. And learning new Maths is not as directly applicable or as valuable as learning new software domains, but can open up new avenues you wouldn't otherwise have, that learning "yet more programming languages" or similar won't. (It's still more valuable to work directly on relevant things, of course, but sometimes it's fun to broaden your scope in relevant ways.)
I've never used Python, but I like Haskell for the kinds of problems where I'm trying to do something complicated and a little confusing, and I want the compiler to let me know if I've asked it to do something that doesn't make sense. That sort of thing can save a lot of time in the long run, and I usually feel good that the resulting software is reasonably robust. I expect a good Python programmer could write an equivalent program, it's just not the way I like to work.
Libraries are of course often a deciding factor, and so are performance requirements.
It takes some esoteric innovative features to make a difference, but that will also cut both ways; it'll make it harder to hire for and scale, too.
I'm a pretty capable "generalist software engineer", but what keeps me employed is expertise in virtualization and high-performance virtual networking. Prior to that, it was expertise in distributed systems and a knack for debugging distributed failures. The history here is relevant: later in the thread Thomas points out that you can switch domains. My time spent in distributed systems with multi-millisecond quorum periods is more or less directly applicable to debugging synchronization issues at CPU clock speeds. The tools used to observe the issue are different, but the reasoning process for untangling the set of plausible partial orderings is the same.
Specialize in something valuable. Continuously evaluate the next most valuable skill to acquire based on where you are and where you want to be.
There's a pretty common trend in GameDev -> Embedded Systems | High-Speed Trading.
The skillsets are very similar while the domains are quite different.
Not everyone's a climber, and being in "a dead-end job" isn't really a bad thing if you're not ambitious.
I'd just like to spread the message that it is absolutely OK to plateau. You don't need to be a renowned domain expert, you don't need to go into management, you don't need to start your own company... if you're happy where you are, don't sweat it.
Not saying this will definitely happen, but it's a risk, and you don't need to be a ladder-climber to protect against it.
I think most people would probably do better if they just stopped allowing their expenses to grow with their income, and saved and invested more, both in equities and in forms of passive income (like rental real estate). That way, not only are you insulated from shifts in the industry around your primary employment, but you can also probably retire early.
To me, the real perk of working in tech is not that there are so many jobs that I can find one that pays incredibly well; it's that I can find one that pays nicely but aside from that has some other perks.
A lot of friends and workmates have told me that I could be working in a better paying place. After looking into quite a few job offers, my conclusion is that for now I'll stay in my 8-to-4 job that pays me twice as much as the average in my area (average in the sense of all the population, not just tech jobs, obviously). I used to have a high-paying, high-stress job; I don't want to go back to that, it's not worth the money.
I must add that I'm EU-based. I understand that in the US, with much less of a safety net, maybe optimizing for money is a more attractive idea. I can't really say for sure.
For example: I've spent a large chunk of my free time over the last few months learning how to typeset signs in fansubs and make them look good, something that has no economic utility whatsoever, but makes me really happy when I typeset something that looks good. Yeah, sure, I could have spent that time furthering my career by developing a marketable tech skill, writing tools for my company on my own time to impress management, or writing an open-source piece of software to pad my GitHub portfolio, but I'd rather do something fun for myself than do any of those.
The problem in software development that current way how company financing is set up is that you do not need domain expert to make it.
In other words, it is nearly impossible to be hired as domain expert because very small number of startups needs one: just do some "growth hacking", get to YC, and raise money. Maybe founders need to domain expert to get to YC but they you will kick him or her out as soon as they get "YC stamp".
And this is reason why there is no database experts in all these new database startups, there is no security experts in security startups, nobody can figure out to make even half decent Dropbox clone, etc. The list goes on.
And there is also ageism: all "domain experts" are 30 something.
Some people get to this point and never step back on the gas, because it seems like things are going to be great forever. Once people get too comfortable, it's only a matter of time until it bites them.
IMO it's important to continually reinvent yourself throughout your career, to keep things interesting, but also to always put you in the best position to take whatever the next step is. If you wait until it's time to take that next step before figuring out where you want to go, it's too late.
But I don't agree that having constant side projects and learning the new cool language/framework du jour is necessarily valuable.
I think that there's just too many technologies out there, moving too fast, to be proficient in a significant fraction of them, even if you invest all your time in learning them.
I think there's more value in learning how to do engineering well in one language than learning how to do basic stuff in 20 of them. Because the skills and knowledge of going deeper transfer much better than knowing syntax, etc - stuff you can always pick up for the next job once you know what it is.
Is it a rule, though? I hope not. I'm a 40-year-old generalist, and so far I haven't had problems finding work. Sure, I'm not going to get a half-mil offer from anyone, but that was never the case at any age.
i've been wondering if i should pick up something entirely different on the side (e.g. machine learning for broad applicability, though that does not excite me at all), or to instead diversify from my current niche but learn to write tools across a broader variety of languages and ecosystems. it's scary sometimes how the entire programming ecosystem can change overnight and leave once-valuable skills in the dust.
Some fields might be locked up by postgrads (crypto engineering is not one of them, and I doubt language tooling is either). You should check before you invest too much in it. But I would advise against being spooked by academic credentials. Good places to work care more about conversance than credentials, and you get to draft off of all the research work the academics publish before moving into industry to compete with you.
I've been in this industry since copper and telco was hot shit.. I went from doing Telco programming > IT > Web Development > scalable enterprise applications.
Why bother stressing over something you have no control over? Do what you love and the money follows.
i do have some hope that i've gotten into my current niche (static type inference for dynamic languages) reasonably early in its life cycle, but if i'm wrong i don't want to have not kept up with the larger world at least to some extent.
(to be precise, i'm not so much worried about being able to transition my skills to something else, as about not being given a chance to do so unless i can already demonstrate the new skillset people are looking for.)
I vehemently disagree with this. Kludging together an unmaintainable codebase that barely meets the spec might be easy. Aiming for a notably higher level quality beyond that is NOT an easy thing at all.
Obviously, the nature of this discussion depends on the complexity of the task at hand. There are many profitable ventures with low levels of complexity in the software implementation, and your comment may hold true for such scenarios.
However, there also many ventures that require significantly complex software to meet their goals, and will require skilled developers to have any hope of not collapsing under the weight of shoddy implementations and impenetrable abstractions.
On one of my previous jobs one problem domain was some area of theoretical physics. There was a guy with brilliant understanding of the problem domain. He had couple of decades of experience teaching physics in the country’s best university. He could explain very hard problems in easy to understand way. The company filed multiple patents with the amazing stuff he invented.
It didn’t help the guy to write code. It wasn’t good. Not only unmaintainable, also slow and unstable (not all algorithms work equally well when mathematically-real numbers are substituted by software-real IEEE754 floats). We ended up just throwing out whatever he coded, and instead learned enough problem domain from him to be able to understand and implement what he actually was trying to do.
Developing transferrable strategic and architectural skills between verticals (client facing, or technologies) is a key aspect that I see being missed with programmers.
No framework will make or keep your career, nor will a specific language or piece of tech. Knowing how to apply technology well, in a way that doesn't need to be entirely thrown away every 2-3 years is invaluable, and as a result increases a developers value.
Mobile app development is becoming quite clear cut, one only needs to read the docs and know the APIs.
I used to know this guy who knew the ins and outs of the health club industry, and he could program a little. He made a lot of money. I think this is more in line with what tptacek is describing.
Being an expert in iOS is more than just knowing how to program in Swift/Objective-C. It encompasses a deeper understanding of mobile development and the apple ecosystem.
True senior level iOS people will understand core user interface design principles as they relate to mobile, and will be intimately aware of Apple's Human Interface Guidelines. They'll understand mobile accessibility and be deeply familiar with the myriad of tools in iOS. They'll understand internationalization and how to design an application to support various calendar systems, and text layout paradigms. They know the ins-and-outs of Apple's provisioning and certificate system and can quickly sniff out configuration issues. They understand how important testing is and how well supported it is. They know how to setup automated builds using xcodebuild, and how to support automatic unit and ui tests.
The list goes on and on. Certainly one could just be an app developer and stop there. But being a mobile domain expert in my opinion is a much more valuable albeit niche position.
It's better to be good at mobile than to be good at nothing at all, sure. But I'd be really careful about picking a specialty that amounts to "my domain of expertise is AppKit", or "my domain of expertise is Scala", or anything like that.
You could claim that software security is "just programming" for instance... But you likely wouldn't, because it requires a level of understanding that is especially deep, and niche. It doesn't really matter that a security expert likely reads a ton of code, and writes some too, it's that they understand a subset of the industry to a high enough degree that they stand out among their peers.
I propose that you can be a mobile expert, just as you could be a VR expert for example. At the end of the day it's more then just programming, it's a combination of hardware / software / tools that retains a degree of uniqueness.
Yes: you can specialize in being a really good programmer for a particular platform. I have a lot of friends who specialize in being Serious Engineers, who understand how to get interesting and important things done with lenses and how to effectively get a whole project away from null pointers and into monadic error checking and how to use type systems to prove programs mostly correct as they're being written.
That is a real specialty. It takes real skill and investment of time to get to the point of being expert with that stuff. I'm not diminishing the value of being an expert software developer.
I'm just saying that basket of specialties is unlikely to command a very high wage premium, not because it's unspecialized, but because it's hard to fashion a value proposition for it relative to your competition in the market.
There will be times during which particular platforms or languages will get hot, and it'll be good to be expert with them. But (a) those phenomenon will be more transient than other problem domains, and (b) the wage premium for being specialized in them will still probably be lower than the wage premiums for the other domains.
At any rate: as long as you're thinking about the domains you're specializing in and not a career as a "programmer" or a "builder of software", we can probably agree to disagree about the rest of the details.
I agree with this entirely. But it makes me realize that perhaps I haven't explained my position well.
I would agree that being a mobile expert is likely going to be LESS lucrative than being, say, an expert in recombinant DNA technology with a fluent C++ background and a PHD in machine learning.
The only issue I see with advising people to invest tons of time and money into niche fields, is exactly what others have pointed out in this thread - You can't really predict what's going to be valuable in 10 years.
So take your pick - Go the safe route and become an expert in a more general field like, for example, mobile. Or pick something incredibly niche, which will shrink your competitor pool and increase your relative value proposition. A classic risk/reward scenario if I've ever seen one.
I don't think this is all that safe, really. What we consider "mobile" development has really only been around for about a decade with the releases of iOS and Android. (Sure, there were mobile platforms before that, but most people didn't really care about them, and specializing in them would likely limit your options, not expand them.) In that time, mobile development has gotten nothing if not easier. So, over time, being a "mobile dev" will become less and less of an important specialization. More people will do it, and the frameworks around it (both those released by Apple/Google, and third-party things on top of it), will make it easier and easier. By that point you might be an "elite" mobile developer who can do certain things on mobile that most others cannot, but the market for that level of specialization will likely also be small.
To make matters worse, developers will disagree with each other about what the Right Answers are for a given platform or language or engineering approach.
Cryptographers (as a counterexample) also disagree with each other about the Right Answers in crypto. But serious crypto people are generally competing for jobs only against other serious crypto people, and there aren't that many of them.
Moreover, the general domain of cryptography isn't going anywhere. But look at Objective C, or Windows Mobile.
Finally, proving value as a subject matter expert on a language or platform depends on companies having hiring processes that reliably spot expertise. But, of course, hiring processes do a terrible job at that.
The fundamental rule of career progression in the technology industry is to position yourself as a domain expert in something more specific than "programming".
It's very difficult to "own" "programming" in a way where you can use scarcity to drive integer multiple increases of the median salary. It is not as difficult to "own" distributed systems, sensor fusion, software security, or OS kernels.
As a general rule, the people getting outlandish half-mil-a-year offers from big tech companies are domain experts, and the people on Twitter shocked and upset that they're never seeing these offers are either (a) generalists or (b) people who have chosen to specialize on the craft of programming, rather than a domain to which programming can be applied.
That's not a value judgement! There are a lot of ways in which refining a true engineering discipline for assembling software is harder than kernel development or fast computer vision processing. It's just a statement about supply and demand, and about ease of demonstrating value.