Good programmers I know also overestimate the skill needed to earn high salary in this job. You don't have to go up the learning curve much; these days, you just learn yourself JS a little bit and go for a webdev job, making shit code and still earning more than most people in a given country.
> But wondering why the wealthiest corporations in the world pay their workers high salaries is perhaps like wondering why water is wet. Software has a low marginal cost, and the rest is basic incentives for the corporations.
Nah, that's like wondering why is this ice block sitting on a hot plate and still solid. The answer is: because it just got put there, and it'll melt in a moment. So too, will end high salaries, as most low-hanging fruits get eaten by software, made by mass-produced cohort of programmers.
Our industry has its share of cycles, but this, in my view, is largely wishful thinking on the part of people. Nothing wrong with optimism but...
Every 5-10 years there's a "technical shift" that forces everyone to reevaluate how they build software or more importantly what they build, and the race starts all over again. The ice block is removed from the hot plate is replaced by a bigger, colder block of ice. And when these technical shifts aren't taking place, the bar for what constitutes as "good software" inches upward.
If your standards for acceptable software were frozen in time in 1985: using modern hardware and software toolchains, you could accomplish in one day what used to take a small team an entire month. But if I delivered, say, what passed for a "good enough" database in 1985, it would resemble someone's 200-level CS final project rather than a commercially-viable piece of software.
Many of the underlying problems and solutions exist for decades. Database systems you mention are a good example of this.
While there are some key concepts to things like databases, the fact remains that your 1985 database would not be considered sufficient in todays world: It would have too many limitations, lack features we now take for granted, would not scale to modern data requirements, etc. Supporting all that "modern" functionality is non-trivial and requires a huge amount of effort. You can't just say "Well, we figured out space and computationally-efficient hashing, so relational databases are well on their way to being feature-complete"
There's a reason we haven't stuck with 1.0 on our platforms, and it's not just security or a desire for a bigger version number: New demands required new functionality and new ways of building things.
iOS was basically AppKit, so anyone already developing for the Mac knew most of what they needed to know to develop for iPhone.
Pretty much every programming innovation is incremental, and doesn't require throwing out all of your previous knowledge and starting over.
Maybe. But AppKit was not the Mac Toolbox.
When my career began being good at memory management was a skill to be proud of. I would say now, being good at concurrency is a skill to be proud of.
I don't really have to worry about memory management any longer but didn't worry about threading when I started my career.
As I see the younger generation entering the programming field I wonder in what ways the craft will be different when they've had a few decades under their belt.
Will parameter tuning datasets for machine learning be the coveted skill? Who knows.
But the demand for developing in AppKit suddenly increased by orders of magnitude.
That's the sort of shift in the environment that the grandparent is talking about. Fundamental CS tech was arguably better in the 1970s and 1980s, because it moved more slowly and you had time to get the details right. That doesn't matter if you're building say a mobile Ethereum wallet in 2018, because you're building for the user expectations of today, they don't care about data integrity or security as long as it doesn't fail during the period where they're deciding which tech to use, and software that solves the problem (poorly) now is better than software that doesn't exist.
I believe you are a victim of survivorship bias.
There was plenty of shitty software in the 70s and 80s. The difference between then and now is that we haven't been able to wait for 4 decades, to see what software of 2018 stood the test of time.
In the 1980s there was a lot of "foundational research" (poorly re-inventing the wheel) for microcomputer people who did not know about the work done on large computers in the 1960s. Move fast and break things was also very much a thing for microcomputer manufacturers and most microcomputer software vendors. Look at how many releases software packages went through, and at what rate.
That has always been the case and is about as good a one-line summary of the software industry as I can think of.
The "modern" database systems are now going back to the exact design principles that the books you refer to solved long time ago. There is tons of research, dissertations,.. that focuses on this from decades ago.
Its just now that the new systems realize that these problems actually exist.
If you dont know the history of a certain field and what came out, you repeat and make the same mistakes again. This seems to also apply to software engineering.
A great example of this is the evolution of FB/Google/Amazon. Portions of their core tech have been completely re-written over the years for marginal gain, but there is a large premium to being the best in tech.
In other parts of the industry every new cycle enables some new area of tech, and those marginal gains become the minimum bar for entry. e.g. Deep Learning and Computer Vision, distributed systems and cloud computing/SaaS.
Haha in 1985 Amiga had a multitasking GUI desktop in 512k and 7Mhz. Now we have millions of times more computational power and struggle to deliver an experience as crisp and satisfying.
I wish people still made software like it was 1985, that actually used the full power of the hardware for something useful...
- No process or kernel security (processes could rewrite kernel code)
- Processes could effectively disable the scheduler
- Supported only a single, fixed address space: both a massive limitation and a performance hack that made the system bearable to use
- Single-user
- No security model
There are embedded applications these days where not having these features are deak-breakers. Let me assure you: if you re-implemented the original Amiga OS feature set it too will be screaming fast. The tricky part is keeping it fast once you start adding protection and additional functionality on top of it.
And largely what happened when you tried to implement more complicated applications on top of these primitive systems is that they would crash the entire system, constantly.
Well the PC wasn’t backward compatible with CP/M so that’s an odd critique to level at the Amiga.
Backwards compatible didn't matter in the mobile world, or for those screaming for ARM laptops.
Sometimes we can't just have nice things.
That’s chicken-and-egg. Why do modern apps with actually quite simple functionality need all this vast power, the GHz and Gb? Because it’s there. Why does software crash? Because the OS let’s it with nothing more than inconvenience to the user.
Amigas were actually quite usable, they were stable enough for complex software to be developed and real work done. Same for ST’s.
so, your software concepts from 85 would be overkill today, not lacking.
Today I'm working on systems with memory latencies from l0 cache all the way to tape storage. And it's getting worse.
Edit:typo
Also modern non-indie games are orders of magnitude larger and intricate than 80's games, and have about the same distrubution of quality/bugs. They're now made by teams of size 10-1000 rather than 1-10 though.
There was one variation of Windows in 1985. (And I remember installing it!)
Still, in 1985, even DOS apps had to target varied environments: mono, CGA, EGA, Tandy graphics, different memory configurations, 8088 or 286, printer drivers...
I wish it inched upward... So much software now-days is bloated crap.
Even if you have library support to hand hold your budget coders, even if you use a lot of them, even if you give them all the time in the world, they will produce more complicated, less coherent, less stable, buggier, and harder to modify, improve, or iterate results than better coders who understand the problem better.
That means that no matter how little you pay up front you end up paying more in the long run throwing more man hours and money at fixing the mess made. A good mess is easier to maintain and improve and costs less over time. A mediocre / bad mess takes substantial efforts to maintain and iterate on.
Its also probably a domain impossible problem to remove the ability for any coder to make bad code. If for no other reason that in any programming environment you can never stop someone from iterating by index over a binary searchable tree for a specific element in it, or you can't stop someone from turning a float into a string to truncate the decimal part to then reinterpret the result as an int. But if you don't give them the tools - the integer types, the data structures, access to the bytes in some form or another - you aren't really programming. Someone else did the programming and you are just trying to compose the result. A lot of businesses, like I said, can be sated on that, but its still not programming unless you are in a Turing complete environment, and anyone in such an environment can footgun themselves.
SQL solved that problem a long time ago. You write SELECT, the query optimizer figures out how, using any available indices.
Eventually it has to change (imo), either through companies becoming more scrupulous in their hiring, or through a massive flood of new devs.
It's not that tech, even done by shitty devs, isn't valuable. It's a question of whether the market can control itself, which I'm pretty sure is no.
Stricter hiring on the low-end perhaps, because in FAANG and the companies that copy them, hiring could hardly get any stricter...
There will be a day, but when is hard to say. Thinking it's right around the corner is akin to the belief we're on the cusp of true AI. We're more pessimistic today than we were in the mid 80s. And non-programmers were programming with hypercard, filemaker pro, VBA, etc, back in the 90s.
There are of course former well paying jobs such as old-school front end devs (html/css/sprinkle of js) that are largely commoditized, but that's a given considering the low barrier to entry.
The same highly productive nature per employee is found in virtually every other high paying industry, most of which have not seen the pay for the higher end of those in the industry fall over time.
With these jobs in particular, what I see is that the definition of seniority has shifted to 'knowing the latest tech'.
So a junior dev who's just got to grips with React has become a React Developer, and they are now relatively senior in that field. The experience isn't transferable to other parts of the software stack though, it's too heavily tied up in the browser. So they end up as a super-specialised frontend dev.
It'll pay pretty well until the tech becomes obsolete, unless that kind of person enjoys maintaining legacy code.
It wouldn't surprise me if index funds have caused a decline in earnings for investment bankers.
On the other hand, if you think the software industry has a hard time figuring out (at hiring time) who the high performers are... science is driven by serendipity. Nobody can predict who will find the billion dollar discovery. Not even past performance is a reliable indicator.
So it makes sense to me that the salary spread in science is relatively even. If they could reliably figure out who to dump money on, they would. On the other hand, the FAANG companies clearly believe their hiring practices can select out the high performers... and perhaps they are right? If they're paying 3-4X what everyone else does, they expect to get at least 3-4X the value.
The selection process seems to do a good job of keeping out the lowest tier at least, although we openly acknowledge that we miss a lot of good people as well.
Years ago, I worked someplace where a colleague was tasked with working with another developer on project X. After about 15 minutes it was clear the other developer ... wasn't? A web project, and this person had been employed as a "web developer" for at least several months. Questions like "how does this information in this browser get back to the server?" came up.
Colleague goes to manager and says "I can hit the project deadline, or I can make sure other_dev learns the basics enough to be able to contribute and understand projectX, but I can't do both by the deadline. Can we move the deadline back a few weeks?"
No, and no. Train other_dev and hit deadline.
Deadline was hit, other_dev moved to another project afterwards, and was pretty much as ineffective as before, but colleague was then saddled with this reputation of being a 'bad mentor' because the next team learned other_dev didn't know how things worked. Why the hiring manager wasn't tarnished... who knows?
He was given 2 tasks and he only delivers one result.
I know this sounds... not ideal... but it is what it is.
His manager probably has to operate at the same level of expectation: given 3 tasks by his manager (or director), either you finish all 3 or you're less dependable.
I was motivated because my older brother, and my mom, had already learned how to program, and they were quite excited about it. After getting past a few familiar conceptual hurdles, it became very easy for me to learn programming myself.
People who are only motivated by the money, or under pressure from others, have a harder time, because their curiosity and drive aren't activated. There's some sort of valve that lets the knowledge into your brain, that has to be opened.
For the most part, the people I know who seem to be motivated by money itself are not so desirous of getting rich per se (many are already rich), but are actually interested and curious about money in the way that I was curious about programming.
I don't program for a living today, but my ability to program is definitely a force multiplier for my work. It has either improved my earnings, or improved the continuity and longevity of my career.
may I ask what domain you are working in? Can you give some examples of how you've slipped in some programming knowledge into other job tasks? I love to hear people's anecdotal problem/solution approaches. Was the programming side of it actually slipping in some VBA/chrome extension/javascript or was it more of just an 'analytical' approach taken to a business decision.
I use programming extensively as a problem solving tool, for things like data analysis, modeling, automation of experiments, and prototyping. Almost all modern equipment is electronic and computerized. To be capable of rolling out an MVP on my own, I program.
You will rarely see my computer without a Jupyter notebook on the desktop. ;-)
In addition to working in a computerized field, program code is just a super powerful way to express ideas. And the disciplines of good programming practices (yes, learn them) provide ways to organize the innards of complex things, so they actually have a fighting chance of working and being right. Plus, it's fun.
People who work as full time programmers may make more money than me, but I'm not sure that I can do their jobs. When thinking of any profession, a person should not only look at the cool, fun stuff, or the money, but what the actual daily grind looks like, because that's what you have to survive.
But that is no definition of bubble. Bubble, at a very basic level, means there is a lot of capital flowing within, it has little to do with whether how difficult your job is.
Medical record systems were running on MUMPS (schemaless) and being eventually consistent (records were keyed in from paper forms) long before 1985.
I mean: what would you use Node.js for - in 1985 - even assuming you had access to a system to develop and test stuff made with it?
In 1985, your hypothetical bank would be running VMS, which had asynchronous IO system calls as the default. There was no need to "invent" something like Node.js.
The fat package makes programmers not realize this.
why in 2019 is there no good way to visualize a decision tree? having to install, configure, and get working graphviz feels very hoaky.
I prefer python, but R still does some things really well that the python libraries are just not up to par
1) anything geospatial, including drawing maps. here is a list of projects my students did to give examples: https://pennmusa.github.io/MUSA_801.io/
2) time series
3) linear models, is it so hard to give me a good summary?
If anyone knows of any packages that do these >= R, I'd love to see them :)
You should try https://github.com/parrt/dtreeviz. From Terrence Parr and Vince Grover, released in Sept 2018.
There is a good background article on the problem space and their design iterations here: https://explained.ai/decision-tree-viz/index.html
I highly recommend spending some time in C/C++, Go, or Julia to pydata-first folks that ask.
When I took the ASVAB in high school I scored a 107. That score is too low to become a warrant officer so I had to retake it a couple of years ago for my officer packet and I scored a 129 out of a maximum 130. That puts me in the top 0.1% of testers. I am not smarter or more intelligent than when I was in high school. I do write software though. Every couple of years I look back on my software and algorithms realizing how I continue to improve and see the solutions more clearly.
https://en.wikipedia.org/wiki/Armed_Services_Vocational_Apti...
The best teacher has been helping collegues. I have been programming a lot better ( less errors and even without sometimes running the application when I'm pretty sure), because I think and analyze more upfront than I used to.
Some things come back, but it's rarely related to me ( eg. last moment spec changes).
I do have to watch out, I notice that basing my code on someone else is ok. But, they always have faulty code in hard to test areas. So make testing easier on things that are hard to test is my next motto.
Also, helping others is a pretty huge timesink :(
Ps. Being in the zone does wonders lately
PS2. There was another threat about videolan yesterday. And nobody heard the entire story about https. I gave the VLC developers the benefit of the doubt, not knowing everything off their infrastructure. . A lot of the comments here on HN disagreed with me ( stole silently upvoted though).
Today I saw a blog post about why..
It was infrastructure based...
I can't understand why I was practically the only one with another view in the subject in this community, where developers come together.
FYI:
Comments are in my history. Mostly on the videolan topic. It's all recent
The question the author posed was why programmers are paid that much even when some other paths could seem "harder", which seems valid. Sure not all careers are supposed to be "harder" than programming, but they're not as easy as one'd imagine either.
Though yeah at least for now I don't see the situation abating much. The demand is still going strong. Once the proverbial "flood" of the market happens from new grads, things might get worse. But still if you know what you're doing, you know all the right concepts and skills, you should be able to stay on top of the game. There has always been a saying that the irony of the CS degree is that many people who graduated with the degree can't program, while many who can program didn't need to do a degree at all. I doubt the influx of students trying to study CS would change this situation much. Coding bootcamps have been around for a decade yet they don't seem to change the market equilibrium that much.
My kids have mentioned that they might be interested in a degree in computer science because and I've encouraged them to combine that with a second area of specialization. Programmers are everywhere, but a programmer that also knows chemistry or biology or economics or art history or just about anything stands out.
Really? I have a physics degree with some experience in rocket science, but my most valuable skillset (measured by how much pay I can fetch for it) is plain old software engineering. I don't think I'd be able to leverage my area of specialization to exceed or even match what I can get from FB/LI/G as a generic software engineer.
That's how markets work. You get paid what what the market will bear, not what you "should" make.
My comment was more about the companies though which may form cartels to drive down employee wages. Companies forming cartels is illegal, while unions are legal in many places.
In the US, union workers make between 10% to 30% more than their non-union peers[1].
You're comparing pay across two different economies and only looking at unionization as a variable. It's like wondering why engineer rates in Omaha, Nebraska aren't on par with those in New York, and concluding that it has something to do with differing fire codes.
Correct, but these protections exist (and existed at the time) independent from a union.
> In the US, union workers make between 10% to 30% more than their non-union peers
There's very few high-skill jobs which are commonly unionized. In a market where supply is greater than demand, then yes unions have absolutely shown to improve worker outcomes[1]. I'm not aware of any evidence for markets where demand outstrips supply (like that for skilled software engineers). It's not immediately clear that union protections would be beneficial.
>You're comparing pay across two different economies and only looking at unionization as a variable.
No, I'm simply pointing out that your flippant response to esoterica doesn't actually address the question. If unions are better for workers, why is it that a non-union area !!with a cartel depressing wages!! was still substantially better for workers than a unioned area with no such issue?
Saying "oh the market is different" ignores the question of why the market is different.
[1]: Indeed, that's kind of exactly what happened with this cartel. Facebook wanted to hire skilled engineers, and was willing to pay more, so broke the cartel. That kind of thing won't happen when workers are generally equivalent, but SWEs aren't.
Sure there are. Doctors and actors, to name just a couple. In both cases the "union" actively works to create barriers to entry.
The AMA colludes with medical schools to set artificially-low student body quotas. If you've ever wondered why teaching "XYZ for pre-meds" is such a miserable experience, this is why. You have to earn straight A's to get into med school because there are so many more qualified candidates than openings (but it's not clear to me how, say, art history or algebra-based physics makes you a better doctor).
SAG (the screen actors guild) requires actors to have already performed in a SAG production a a condition of membership. And they also strictly limit the number of non-SAG performers on SAG productions. That chicken-and-egg problem was very intentional
If you've ever taken a macro economics course, you know what effect these actions have on prices.
> I'm not aware of any evidence for markets where demand outstrips supply (like that for skilled software engineers). It's not immediately clear that union protections would be beneficial.
See above. Unions can create a market where demand outstrips supply.
> If unions are better for workers, why is it that a non-union area !!with a cartel depressing wages!! was still substantially better for workers than a unioned area with no such issue? Saying "oh the market is different" ignores the question of why the market is different.
So tell me why professional associations exist, then. Why do doctors form a union to increase wages, if as you say, they would be better off without it?
Neither the SAG nor the AMA are unions in the traditional sense. In many ways, the AMA actively works against worker quality of life (consider that the horrible conditions for med students/residents and the high suicide rate among MDs) to artificially reduce supply.
>Why do doctors form a union to increase wages, if as you say, they would be better off without it?
The AMA is mainly a lobbying organization, not a union. Since a significant percentage of doctors are in private practices or small practices, they don't have representation with the government. So sure, the AMA does collectively bargain with the US Government. But by that same token, since 53% of MDs are self employed, the AMA can't do "normal" union things like set wages, because there's no one to bargain with except the doctors themselves.
And interestingly, the AMA actually admitted that its intentional supply-reduction is hurting the medical industry as a whole. To answer your question, "because they thought it would be better". But in hindsight, they probably weren't.
https://translate.google.com/translate?sl=auto&tl=en&u=https...
I suspect your posting in bad faith here and in the usa SAG minimum rates doesn't effect the higher rates that successful actors get.
Another way to see it, collective bargaining goes both ways, ie both workers and employers will come to a joint agreement. So if we created a FAANG engineers union and created a joint pay-scale for them, then that would basically be equivalent to the non poaching agreement often derided in discussions like this.
Not all union models have sector bargaining and it certainly doesn't work for professional unions - and I am not saying that European unions really get the needs of m&P members and need to change.
In every other aspect of computers, the industry has finally embraced usability as a desirable goal, and not just for end-users.
On my first computer, you had to read a 100-page user manual and learn exactly what commands to type. In my first programming language, you had to manually allocate (and worse, deallocate) memory. With my first database, we used to have to go type VACUUM regularly. None of these is true today.
Yet even though some of the highest paid people in the world are members of unions and have agents to do their negotiating, programmers seem to have latched onto this idea that if you're not making top dollar or have your ideal working conditions, you should "just negotiate better".
Why stop there? Tell programmers they should "just program better", too.
> You can also unionize
Have you ever organized? I don't think you realize how difficult this is, especially without strong support from an existing union. There's a reason unions heap rewards on people who do it.
Existing unions also have great labor lawyers. A common response to even thinking about unionization is getting fired. (That was in the news recently because it happened 4 weeks ago here in Seattle.) Labor laws aren't what they once were, and there's usually no consequence to the company for firing organizers.
Flipside: I can still write software for my first computer without looking anything up, over 30 years after reading those 100 pages. I still know the memory layout, opcodes, assembly etc by heart and it is still the best way today to program that particular computer (which still works in my man cave) today. Yes, today it is all simpler, but the 100 page example I find a plus, not a negative. Maybe you were referring to something but my 100+ page manual was usage and at the same time programming (using was programming beyond the basics) as that was the only way to use the system.
Issue 1: It's very difficult to tell if your contribution got 50% improvement in performance because there were 10 other devs pushing in features and bug fixes. This is the attribution problem
Issue 2: This happens over time. It's very unlikely that your 50% improvement happens every year or month. Because, think for your self, this is compounding with large rates. It grows quickly. 1.5x improvement in 6 cycles (months or years) is 10x. This essentially is the time problem
Issue 3: even if you deliver the results you did, in a large company there's a large bureaucracy and no one person has the ability to increase your salary by that much. This is the control problem.
It’s avtually fairly non-trivial to be able to say with even a modicum of certainty how much value a given developer brings to their company.
I currently write software used by millions of people. Partly because I’m a backend engineer, I have no real idea how much more the company is making due to my direct efforts. Since they keep paying me, I’m assuming it’s a decent multiple of my carrying cost, but I have no way to measure it.
In other words, markets work that way because that's how the bosses and capitalists want it to work, and have so far been successful at thwarting attempts to use the government to change things.
Starting your own company (not self employed contractor) gives a really good perspective on what it means to be owner and employee.
The logic is that programmers tend to produce far more value than they capture -- so that gets captured elsewhere, a lot of it typically by management. Except the value can be hard to quantify when the company is old and so is the software, how much of the value is employee #3701 making fixing a bug that's making the product not work for one customer in one instance vs employees #107, #85, and #150 who in their past team's life created the original version of that system to begin with to make the new customer even consider using it? There's no point to moaning about how much you "should" get paid. Just ask for more if you feel underpaid, but be aware that because of competition and because people usually want to hear what more you'll do to justify it, you're not going to always get it.
How much of the business risk of the enterprise is your top flight programmer assuming? Are her decisions the only ones that make any difference to the increase in profitability as a result of her work? How direct is the line between back room engineer, no matter how good and profit?
The only case where 50% or near it makes sense is for a founder owner who is also the lead talent. Maybe. Because then they are also creating the business opportunity and assuming a big chunk of the risk.
Most SWEs I know are not making more than they bring in profit.
Agree
Some of the smartest people I know work in other domains: biology, chemistry, and even physics. They are sometimes baffled by tasks that seem trivial to me, and I'm under no impression that I'm more intelligent than them. I simply specialized and focused only on programming, while they program to accomplish other tasks in their domain of expertise.
Can this last forever? Of course not, nothing lasts forever. But wondering why the wealthiest corporations in the world pay their workers high salaries is perhaps like wondering why water is wet. Software has a low marginal cost, and the rest is basic incentives for the corporations.