Application engineering is getting easier and easier to the point where people no longer need a degree or even extensive training to be able to do application engineering work. Employers are aware of this, startups especially, and often prefer unskilled individuals (or people with only domain expertise) who are willing to work extra hard to teach themselves the skills on the job. They often pay these people less as well. This was explicitly stated as a good strategy by a ycombinator partner, and the executives of several successful, well-known, mid-size companies.
Honestly I hope this isn't true, maybe someone else can shed some light on this.
What will change is who is doing the work, how many people are employed, and how much they're compensated for their work.
I also want to mention that a number of my friends have gotten job offers from Google, and they're much lower than the offers other friends received just a year ago for similar positions (at least $20k less).
What? It's vastly easier. 10 years ago, you'd often buy servers. Getting a HA DB required a bit of knowledge (especially if you weren't using SQL Server's easy cluster stuff). Backups were another hassle.
Now? Click "New Database" and select a performance and HA level. Done. Look at StackExchange - they just scaled the DB up cause RAM is cheap enough to put a TB in a server. That's vastly easier than dealing with scaling out. And even to scale out, "cloud" DBs like SQL Azure have sharding and auto-scale built-in.
On top of that, servers are MUCH faster. Look at the RPS that some sites are claiming. They simply aren't that high. 1000 RPS? 10 years ago that required some planning. Now, so long you don't do dumb things, it's not an issue. (Source: I ran a system that did hundreds of millions of transactions a day, and the hardware requirements were rather light. Each transaction ended up in an ACID DB UPDATE, in addition to archival, auditing, HTTP requests, etc.)
It's simply incorrect to say it's not easier to scale today. It is.
There has never been a positive correlation between the ability to build/maintain applications and having a degree or going through extensive training. The job has always and will always consist in learning things by yourself, as and when you need them. If someone needs hand-holding in order to learn a new language, a new API, a new tool, or anything else that we use, he is simply not suitable for the job. You cannot count on receiving "training", because with relatively new technology, there may initially not even be any training around. In other words, there is rather a negative correlation between seeking knowledge in degrees or extensive training, and the ability to do the job. The person may simply never have learned to figure things out by himself. This would entirely disqualify him for the job. Furthermore, application engineering is only getting easier and easier for a certain group of people. For everybody else, it is getting more and more difficult.
The difference is that computer networking has become more seamless and ubiquitous, and so the process that you describe of hiring unskilled (by traditional measures) workers doing domain-specific programming work for low pay has become a more rational one to do, because of information accessibility.
So, all the "two week programing camp" developers launch their 12-hours-from-idea-to-launch apps on top of the work of _actually_ skilled people, and all the reward goes to the final layer, not the underlying support.
Right now we're in an environment where, if you do build a better mouse trap, you get $1 while people using your better mouse trap to build Social Mice Death Sphere get $1 million just for adding fancy graphics on top of other people's work.
Is the only reason our entire zero-reward open source system stays alive is because nerds are economically irrational and just want to have fun while turning a blind eye to being exploited?
I would say no, because at least to me it seems like those who are just building applications by "gluing libraries and APIs" need to be compensated more because their jobs are much less interesting both in terms of the actual work and also to future potential employers.
I would definitely take a pay cut if I were to be allowed to work on something more interesting provided I still have enough to survive.
For those "nerds", why should it bother them that other people are profiting from their work? If anything I would think it should make them happy to know that what they built is helping others all the while keeping them entertained.
I think it's disingenuous to think every open source software component should be a VC funded company operating off support contracts to get any reward. That's essentiality the only avenue to "greatness" right now.
In a proper world, VCs would recognize all their darling, bespoke, hand-picked companies depend on the public infrastructure of open source (and unaffiliated) developers to super accelerate the growth of their portfolio companies.
In a sane world, VCs would see it's in their best interest to create a "public benefit" pool of cash where they fund open source developers to create, maintain, and grow open source projects/platforms at full time senior-engineer salaries. Such a pool of cash (contributed to by multiple VC firms) would be administered by a neutral third party (to avoid nepotism as VCs are wont to do) and have "board-like" oversight and project prioritization (maybe each portfolio company gets X votes towards ranking which features to create next) so resources (cash/salary/infrastructure) go towards projects companies are using and needing to grow.
A different option is to encourage VC-funded companies to set aside, say, 5% to 15% of each round for hiring full time open source developers. Open source developers working at a company would be loyal to their open source projects first and foremost, but would accept direction and feature prioritization from the sponsoring company.
Just because something isn't a company doesn't mean it's not valuable and doesn't deserve compensation for services rendered, but in the current environment that's the only way forward until more fundamental change appears.
1. Fun is it's own reward of course.
2. Producing open source code you get easy marketing and no risk. Try to sell the same thing requires a whole new skill-set, capital, and you will probably end up with the $0 you'd get from open-sourcing it anyway.
3. Not having a boss (or customer) makes it more fun and allows you to be super-opinionated about how you make the framework.
4. I agree though in principle nerds should be more mercenary. For non-fun stuff, get your employer to pay you to write open-source code, instead of doing it on Saturday at your own expense. For fun-stuff ... well it's fun! Try to make a $ out of it somehow.
It seems that machines are much faster at learning how to do low level stuff(compilers, generic problem solvers, etc) - than they are at doing the high level stuff - understanding the problem and build a high level spec/system for the solution.
And of course we're becoming more capable of letting the domain users enter more and more details of the system themselves, either by visual programming, scripting or giving data to machine learning systems, etc.
So if those trends are to continue , professional programmers will specialize in creating more and more , highly abstract generic solutions.
Low-level optimizations are the least automated process I can think of.
The share of programmers who are doing high level stuff(understanding users and encoding that ) versus those who optimize the low-level is always growing.
Maybe it's ~90% today , vs less than 10% in the early years of computing when every byte counted and systems were less complex.
> Low-level optimizations are the least automated process I can think of.
And compilers automate low-level optimizations every day.
I try to avoid the word "System(s)" because of confusion between what we're talking about, system administrators, system architects, and control systems folks. But if you're certain the person you're talking to uses the same definition then it's fine.
If you're the first kind of programmer, do some programming with some of the tools that interest you to learn new things and broaden your experience/understanding. If you're the 2nd kind, pick and area that interests you and roll your own. It doesn't matter if there is a big commercial code or even a big open source version out there, chances are if the right person does their homework they can build a better mouse trap. Just because something is big doesn't mean it's the best that can be, some things just have a lot of momentum.
But no, I don't really see much changing other than the list of things people build and then glue together. That basic pattern (build tools, apply tools) seems to be very persistent. Rather than worry about the future, you should open your eyes to just how wide the field is today.