This mindset is how we end up with layers upon layers of badly designed and buggy software that underpins almost every aspect of modern life.
Just apply the same reasoning to other areas: Would you want to drive in a bus with a bus driver who just barely got his driver's license? Would you want to use a bridge designed by an architect who knows just enough to complete the job and does not care a bit for more?
It does not take mastery to write one-off scripts that "get the job done". However, they will probably not be a general purpose solution of the problem at hand, ignore corner cases, contain bugs… and the real test comes when requirements inevitably change.
If you want to write dependable, high-quality, maintainable, reusable software you better know more than the bare minimum.
The Colonial Pipeline ransomware wasn't life threatening. These things happen because of bad* developers and zero accountability from developers over publishers to software owners. As it is now pretty much all commercial software is "first time waiter" quality. Granted it isn't all on the actual developers but on the whole pipeline(!). Most software is on the "let's use an anchor to stop our newly developed car because we have used anchors for decades and understand them better than drum brakes" level of quality.
* Bad code because of a lack of understanding and/or time constraints. But developers don't mind coding bad code (not enough to refuse at least). Developers need to be more like doctors and engineers. Accountability and proven skill matters and would force managements hand like at hospitals and building projects.
1)
Everyone needs to pump out all of their bad code before they get to their decent code. Imagine your lifetime array of code, it will look like this [bad, bad, ...(lots of bad), bad, good, bad, good, good, ...(lots of good), good]. As you can see, you’ll have to pop (poop) all the bad out before you get to good. Some people have a smaller length array due to other factors (talent), but even so, there is bad in front of the array.
What are the implications of this? That should be obvious. This industry hires people straight out of college or people in their mid 20s to be project leads. You do the math.
2)
Lack of suffering. Many people haven’t had to toil in someone else’s codebase. Many people are given green-field projects that are scrapped quickly, at which point the business (or another business) gives a brand new project. Constantly building shit from scratch by yourself means you don’t understand pain. Go work in someone else’s garbage app to feel pain. Then you will rethink what good code is. Good code is not painful, and the definition of ‘not painful’ will be obvious to the survivors of pain.
Solution:
Continue to crap out your bad code in non impactful areas of the codebase and avoid crapping in critical parts. You have to crap somewhere, and that is understandable. Lastly, don’t turn down the experience of working in someone else’s labyrinth. The experience is valuable.
We aim for fast MVPs, fast iterations, "failing fast", "moving fast", "delivering value quickly", generally preferring speed over quality, agility over "waterfall" planning, continuous delivery over batched releases and, well, often accruing tech debt over shipping features later.
One can argue that "fail fast" should not be about code & feature quality but rather about determining proper scope, splitting features sensibly etc. But then we still have deadlines to meet and code quality suffers.
What I'm saying that this kind of fast-moving environment actively discourages traditional (like doctors & civil engineers) engineering practices.
You might have lots of (unit|integration|E2E) tests but the company may miss practices like proper security reviews and code won't be reviewed, tested and approved by multiple people like, say, bridge building plans are.
This problem mostly comes up not with your usual web app but when critical systems are affected (like in the aerospace industry or industrial systems). But through supply chain attacks, nearly any system can be vulnerable these days.
We should definitely have better regulation on how software is made.
But iterative development and fail-fast are independent of quality. If a company skips security reviews and is fine with technical debt because "it works, just ship it", what makes you think they would do security reviews or care about better-than-minimum code quality when following a waterfall system
Stop equating programming and bus drivers or waiters. They aren't equal. Programming is difficult in its depth and breadth in a way that waiting tables is not.
Stop treating all programming tasks as equal. They are not. Most of the easy ones are only easy because they run on top of all the difficult programs. That is the goal of the difficult programs, to make it possible for laymen to do work that would require an expert for certain repetitive tasks if the underlying program did not exist.
Most people would not be capable of getting Hello World to run without the OS and high-level abstractions that take expertise to develop. And making that possible requires many many experts in many different kinds of programming.
Stepping off of the single thread and single machine model into concurrent and distributed computing to enable even more laymen to make stuff takes even more experts.
There is a meme in the industry that people shouldn't strive for expertise. We still need more experts, and you become an expert through practice while still a layman. Nothing would be worse off if we had more experts capable of deciding when to use something extensible off of the shelf to enable laymen to do simple tasks, and when to build a platform to eventually enable that task to be done by a layman.
A lot of things would be worse off if we encouraged all people to stop at the layman level. We should encourage everyone to at least become a layman for their own enrichment. But we should also encourage laymen to strive for expertise to lower the barriers to entry to produce more laymen capable of doing simple tasks.
Also there are some great thoughts in this thread on why there is such a shocking amount of spaghetti code out there and I like the ransomware note also. Thought provoking. Too bad I'm such a terrible programmer.
No, it’s not. Changing requirements by adding features is by far the biggest cause of badly designed and buggy software. I’ve worked on multiple teams where everyone was fully committed and highly skilled, and it didn’t magically fix the problems of software. It wasn’t any better than working on teams of people who were less interested. Our inability to stop adding features is what kills us, and this inability is actually stronger with people who think they’ve ‘mastered’ the art of programming than with people who can finish a one-off task.
> they will probably not be a general purpose solution of the problem at hand
Ironically, perhaps, in my decades of professional programming, the number one biggest waste of money I’ve seen is people over-engineering something under the banner of making something “general purpose” that didn’t need to be. I’ve watched a couple of different teams of very smart people waste literally tens of millions of dollars by deciding to rewrite something that didn’t need rewriting, and dramatically overestimate their ability to finish it in a reasonable amount of time and avoid the same mistakes they made the first time.
> If you want to write dependable, high-quality, maintainable, reusable software you better know more than the bare minimum.
This sounds good in theory, but is specious. The real way to get high quality software is to define what that means and stick to it ruthlessly. You don’t need to know a lot about programming. You need a management that is okay lengthening schedules to make room for testing. You need a CEO who is okay with saying no to customer demands for features that the competition has. You need programmers who know when to stop programming and when to avoid rationalizing their ‘general purpose’ solution that solves problems that don’t actually exist.
Good luck in your search for high quality software, it’s very very difficult to find a team who can commit to it, and there are good reasons why: it’s extremely expensive.
You nailed it exactly. High quality engineering requires costly business practices that most companies don’t need, so the tradeoff is easy to decide. This is why organizations that do high quality engineering are well funded and are very restrictive about what kinds of programming features are allowed. NASA published a famous essay about how to write safety critical code that disallows use of dynamic memory allocation. https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Dev...
Or any of those things you mention? I normally just trust that the bus company has hired a guy with a license.
How often are you driven around by bus drivers who just finished their learner's permit?
I don't think this part is useful. If you're writing directly code for an application you don't usually need to be the best, and your errors can be handled by layers of supervision/QA. If you're writing internal libraries and tools, you need to be a better programmer. If you're in one of the let's say Rust compiler groups, you will need to really know your stuff. But even the Rust compiler has issues for newcommers that aren't too hard.
I don't think comparing software engineering to bus drivers and bridge architects leads to useful insight. I do agree with the rest of your post though. A part of my day job is to develop and maintain applications made by people that didn't/couldn't program "properly" (shadow IT). Everything more or less work and the end users are satisfied, but I often fear that one day something will depend on one of those applications. On the other hand taking the time to do everything "properly" would lead to a lot less experimentations and/or use a lot more resources. It's hard to find a good balance.
No, I think it is little bit different.
"Most people think they will get a job and then will become experts after some time of performing the job"
And they are right. But they are mistaken what kind of expert they become -- they are becoming experts at what they are doing which means, if they are mindlessly repeating same things they are becoming experts at mindlessly repeating same things.
That's a great way to explain it. Totally stealing that!
The number of jobs that benefit from mastery of the craft is essentially equal to the number of jobs.
I do not enjoy finding the work of a bunch of people that studied enough to get hired and no more.
I think this is true. Nobody disputes that.
Imagine you'd hire Torvalds to debug and fix why your Wordpress does not work. I truly think he'd do an excellent job figuring out that AWS has a firewall that's blocking the connection from your WP node to your MySQL node...
Somehow, the example above feels bizarre and yet these are the types of issues a lot of architects, consultants, and other highly-paid job positions deal with on a daily basis.
> I do not enjoy finding the work of a bunch of people that studied enough to get hired and no more.
This sentence makes no sense to me. I'd say you "find the work of people that studied enough to get hired and no more" possibly just slightly more often as you "find work of people who never wrote anything public until they mastered everything they could, from building computers from NAND gates up to AI/ML".
Also as someone who has had to hire a team of programmers in the last 18 months, it's clear there's an industry of snake-oil salesmen willing to promise to teach you to be employable in 3 months or less, and they're not benefitting the industry in need of good programmers, or the people paying them thousands of dollars in hopes of improving their employment prospects.
A 4 year degree is not necessary to become an employable programmer, but imho what we need more of are 2-year programs which properly teach the fundamentals of computer science alongside practical skills, not 6 week or 3 month bootcamps which only teach the bare minimum.
Not sure about this though:
> I would say that the number of jobs that require the mastery of the craft is not large.
Perhaps the number of jobs that require mastery is less well known? i.e. Not advertised as often / not vacant as often.
I suppose I'm splitting hairs here in one's definition of "not large", by in my circles (~25 years of dev working experience) there are quite a lot of "jobs" requiring mastery, but they certainly do become available less often (or rather, they're usually quietly filled without much public notice).
The job market has to roughly follow this demographic trend, otherwise companies would fall behind competitors.
[1] https://insights.stackoverflow.com/survey/2020#developer-pro...
The rub is that this all depends on what you find personally rewarding. If what leads to a rewarding life to you is having a good job, then absolutely, you probably don't need more than a certain amount of depth and knowledge on the subject.
But if depth and knowledge on the subject is what you find personally rewarding, then suddenly putting in all this work makes sense. For many, lifelong learning is what is most rewarding and brings most joy, which one might argue is the most important thing as long as you have a sufficient income.
If you do want to frame it in terms of achievement, it's also obvious that many prolific people in their fields were driven by a passion for what they do — the playful nature of Feynman, Shannon and others comes to mind.
I've been programming for a lot more than 10 years, and would barely call myself an expert on anything. 10 years is a decent amount of time to learn to be a decent programmer. As a rule of thumb at least.
10 years is not to "teach yourself programming," it's to "become an expert in programming."
Most people do not want to learn programming to become experts, most people want to learn programming to get a job. After getting a job, some will plateau right away, others will plateau after some time, and others still will keep learning even after years and years.
The problem is "how long until I become employable," not "how long until I become an expert".
The answer to the former is months of deliberate practice. The answer to the latter is [tens of] years of deliberate practice.
Books with titles like "learn C++ in 24 hours" target the former. And, I would say that the number of jobs that require the mastery of the craft is not large.