The only real lesson to be learned is "write for your audience" in my humble opinion.
To fully disprove my point, at what point do we never execute something with code so the distinction is even worth mentioning?
The job of software architects is the role Heidegger ascribes to man when he calls us "shepherds of Being". To understand the world as it is, and find the right abstractions to describe it. To constantly evolve those abstractions toward better ones, clearer ones, to seek out ways to represent things that make solutions seem obvious and inevitable.
A mathematics professor once said to me that as time went by, the definitions in mathematics became more complex while the theorems became simpler. "Why is that?", I asked, and he replied, "It's progress!" This is the sort of progress that the software architect seeks as well: to set up the problem so excellently that the solutions are smooth and the process is enlightening.
Great writing, and great code, are first and foremost about great ideas. About brilliant ideas that change how you view the world for the better.
* execution performance time
* build time
* regression frequency
* defect quantity
* code size
* dependency quantity
* test automation coverage
* test automation execution duration
The problem isn’t that we don’t know. The problem is that developers don’t measure things and simultaneously bitch about strong opinions.
I'm not sure the metrics you propose capture what makes good software good.
Perhaps these metrics matter in some small fields but overall most places that need software could care less about most of these, as long as the business objectives are reached. And the business "winning" relies on more than just the code running its software.
Bad measurements are only bad when they disqualify better measurements and better measurements only occur when some measure more directly proves/disproves the desired result.
This is why non-developers believe developers are generally hyper-autistic. For most developers everything must be about clear, easy, simple, safe. These are all super subjective self-serving opinions that don't do anything for the product or the labor that builds that product. Product owners will scream about this, developers will pretend to hear it, will immediately discard it, and then repeat the same insanity where they find comfort.
Step back, take a deep breath, understand that it's not about you or what you want, and finally discard the self-serving circular insanity and only focus on measuring your time to complete a task, time for the application to do things, and frequency of user engagement.
Example: I have a talk with the PO where we agreed on certain features and certain things that do NOT have to work. Often times when I make decisions during development that rest on the asumption that these certain things dont have to work I later on get told to incorporate them anyways. So I have lost a lot of trust in POs or anyone that is not a developer that tells me how a piece of software is supposed to function.
Example 2: I am currently dealing in my department with a case where a Product Manager talked to a Product Owner and they contractually aggreed with a customer to deliver one of our internal development tools, that are absolutely not ready for production or were ever meant for any customer.
Yes I will be "hyper-autistic" about my code because sometimes I do not have a choice.
Suppose I say that good code should have the lowest number of curly braces, or the lowest number of subroutines, or the flattest or deepest object hierarchy. A measure being objective doesn't make it good. So why is test coverage good?
In fact, every single one of the metrics proposed in the parent post is something I have seen gamed to the point of maladaptivness.
* execution performance time - So frequently overly focused on that you can find google hits for "Premature optimization is the root of all evil." I have seen developers spend hours or days saving (sometimes imagined!) run time, where all the time they saved over the life of the product wouldn't add up to the time spent. Extreme pursuit of performance leads to code that is hard to work on - who cares about those milliseconds when critical new features take months to write, or are even impossible?
* build time - It is very easy to diminish build time by destroying critical architecture. I have done it myself. :)
* regression frequency - Insisting on only making very safe changes is how you wind up spending six weeks lobbying change control boards for one line of code.
*defect quantity - In environments that actually track this, people merge issues into a single "fix what's wrong" ticket, degrading the utility of the ticket system. Defect granularity is not actually obvious!
* code size - Obfuscated X contest entries are often very compact, and people who obsess on saving lines and characters can wind up leaning towards that style.
* dependency quantity - Leads to attempts to homebrew encryption.
* test automation coverage - Automated testing ossifies design and architecture, which can paralyze, e.g., an experimental prototype. Full coverage is also costly - time and energy spent maintaining pointless tests can come at the expense of mitigating more realistic risks. I realize I depart from prevailing wisdom in this, but there are times and places when automated testing is simply inappropriate.
* test automation execution duration - Sometimes the right way to write a test is slow.
I'm not disagreeing that these are generally good things to strive for. They are! I'm saying that if you think these things define goodness, each one can lead you to a cursed place. Goodness is subjective, and while objective measures can help you assess it, such measures cannot define it - when and how you use which measures, and when you think they've gone off the rails, is itself a judgement call.
I once inherited a system that was both essential for business operations and a thorn in everyone's side. The guy I inherited it from (and the guy he inherited it from) had taken over a year to learn how to use it. I set about reorganizing, rewriting, documenting, abstracting - all those soft changes in pursuit of clarity and obviousness. They aren't objective, but they do pay off: when I handed the system off to the next guy (and three more after him!), he was off and building on it in a day. That was how I knew I had succeeded! When doing that sort of thing, you do always wonder if what you're writing is clearer for everybody or just you. But surely even the most hardheaded bean counter can see the value of training developers in a day rather than a year. That's good. :)
Goodness is contextual and subjective. I can agree that your goals are generally right, and I can point to circumstances where they're overemphasized or even outright wrong. Sometimes, when the sky is falling, a nasty little bash script that meets none of the usual criteria for "quality" is the best possible thing.
There are people who use subjectivity as a haven for vanity, and build mountains of pointless code in pursuit of some idea of goodness that serves no practical purpose or is even harmful. It is important that we retain our on ability to criticize on subjective grounds, precisely to counter that sort of activity - because you will find it in the land of the objective advocates as well, building mountains of metrics that don't serve any practical purpose either.
Objectivity is no refuge from the necessity of good taste.
That completely misses the point. It’s not about what’s good. It’s not even about what’s better. It’s only about how much better, the distance between theirs and ours. Better is objective only when like aspects of competing items are compared within accepted bounds of precision using evidence.
The interesting thing about measuring stuff isn’t that people are otherwise entirely wrong in their assumptions more than 80% of the time, but that they are typically wrong by one or more orders of magnitude.
>That completely misses the point. It’s not about what’s good
The loss of context here makes me wonder if I am talking to an AI. The comment you originally replied to was,
We know what a good batting looks like but we still can’t say what good code is in any reasonably objective way.
Are we tracking? The original comment claimed that we do not know how to objectively measure goodness in code. You are (apparently) claiming to know how to do it. I am claiming it is impossible even in principle.
In this context, I find your response ("It’s not about what’s good", and the claim that "better" is easier fo measure) bizarre and nonsensical. Like an AI, you seem to have lost track of what we are talking about. We are talking about whether we can objectively measure code being good. It is exactly "about what's good".
Of course it is possible to measure things about code, but equating those measures to goodness relies on artificially constrained circumstances -- like a code golf contest or a PO declaring test coverage a metric to maximize. Athletes find themselves in such constrained circumstances all the time because it is a pursuit dominated by competition and games! It is most obvious in sports like sprinting or powerlifting, which are analogous to something like code golf, but even sports like basketball in which "goodness" is harder to define are heavily artificially constrained such that goodness in a player is about maximizing an objective measure - team score. This might be analogous to a programmer who sees his mission in terms of ticket closed per week. By contrast, programmers in general are usually working in a context in which the quality of what they produce is measured by a lot of complex impacts - on users, on business, on other programmers.
Some of what code needs to accomplish - conceptualizing a problem well, communicating clearly - is inherently subjective, having to do with how it is received by another mind. Programmers (myself included) are generally focusing on these sorts of characteristics when talking about code in isolation, partly because we feel the impacts to ourselves most keenly. But I would contend that there is something deeper and less obvious here - that maximizing this subjective goodness profoundly improves the situation in more objective areas. Well architected code, well communicated code, clear code is resistant to defects in a way that mere test coverage can't accomplish. This is not obvious, but it is deep wisdom arising from experience, and is part of what drives programmers to emphasize the ineffable in their understanding of goodness.
In fact, I was originally going to draw a parallel between being a good basketball player, maximizing team score, and a programmer maximizing business revenue. But I stopped myself, because this is an element of the deep wisdom of good code: focusing on ineffable, subjective excellence is profoundly positive for revenue. It's not something you can measure directly so much as it defines the circumstances the business finds itself in. This observation isn't unique to me - here's a pg essay making a similar point: https://paulgraham.com/avg.html Programmers talking about code being good in this sense may only be building sandcastles in the air - but they also may not be. You must possess the wisdom yourself to tell the difference. But there is certainly more to it than than selfish vanity.
But even leaving that aside, because code must meet a broad array of conflicting demands, optimizing among those depends both on the circumstance and the values held by the people in it. Hence, goodness in code will always have a subjective element, and (in the athletic sphere) is most like saying you are "healthy and fit" or "your best self". You can certainly bring measurements to bear, and we are certainly talking about something real, but there is an inescapable subjective dependence on the value judgement of the judge.
This actually touches on a broader philosophical debate: Are value judgements mere meaningless personal preferences, or are they (often imperfect) attempts to articulate something real? In code, and in life, I believe the subjective is pursuing the real, and moreover that anyone who thinks it's worth arguing about intuitively agrees with this assessment. By contrast, the view that only the objective is real, popular as it has been for the last couple of centuries, and attractive as its promises are, has been increasingly producing absurd results.
As you rightly noted a couple of comments back, what this view does is it smuggles in subjective assumptions. That is, someone operating under this view is going to objectively measure something (like kloc or number of tickets closed or execution time on a test data set) but the selection of what to measure, and the selection of how to value each individual measure of an objective quantity in order to determine overall "goodness", is subjective. The step where they assign meaning to a measurement is a subjective step.
It's interesting to watch the development of "objective" measures in basketball and the dialogue around how to determine if a player is the best, most valuable, etc. over time. Decades ago, the only stats we had were "counting stats" -- points, rebounds, and assists. Steals and blocks came a bit later. There is a correlation between putting up big counting stats and winning games, but it's not as strong as you might naively suppose. Once more sophisticated metrics were developed, something that "subjective" observers had always noticed ("losing player with good stats" is something that was often said about specific players) started to be quantified: some players put up big stats because they're doing inefficient things that result in individual stats at the expense of the team, like taking a high volume of shots even if they're lower percentage shots than a teammate could get on that play, or not contesting an opponent's shot but trying to chase the rebound instead (leading to more opponent scoring but also more personal rebounds over the course of a large number of shots.) In the modern era, advanced stats like PER, VoRP, WS, and BPM are basically more sophisticated models built on top of counting stats that try to scale them and weight them according to regression models. These stats are better, but they still don't capture everything, they only capture things that can be inferred from counting stats. They don't capture things like -- Steph Curry has such strong "shooting gravity" that his teammates often have extra space to shoot because multiple defenders are trying to make it hard for him to get a good look, or Rudy Gobert being on the court changes an opposing team's play selection because he's such a good shot blocker that plays that would lead to a bucket against a different player are leading to him getting a block so teams avoid those plays. Someone insistent on "objective measures" won't even consider these as things to potentially care about unless they have a way to measure them (which, now that we have sophisticated player-movement-tracking, we can actually measure things like how close the nearest defender is on shots by a Curry teammate when he's on court vs off court, or what percentage of opponents' shots are taken in a specific part of the court when Gobert is defending them vs when he isn't. So those measurements are coming online over time.) And, of course, understanding that it means something that Curry's teammates have extra space to shoot, or that Gobert's opponents might not be running their strongest offensive plays because of his shot blocking, puts us in the realm of meaning rather than mere measurement. Knowing to make the value judgment of "it matters how this player is impacting the game in ways we don't have a good numerical measurement of, but that a sophisticated observer who values those things can watch for and give subjective consideration of" puts us in the world of meaning rather than mere measurement.
That seems to be the same issue underlying this discussion. Knowing that conceptualizing a problem well matters -- and that it will profoundly impact the end result in objective areas even though it's not directly measured -- is wisdom.
Just start with the premise that bias isn’t helpful and even less helpful when implicit. Let that determine what to measure and will not be harmed. If such decisions twist you into knots then you are the person qualified to make such decisions.
Determining what counts as "bias" is itself a subjective activity. People who grow up in different cultures have different baselines for what factors matter the most and what factors they consider to be overvalued, undervalued, inappropriately accounted for, and so on. Not just different countries, but different subcultures within the same country (like, my cousins from the farm see a lot of things in society as biased toward big cities, which I never considered because I've lived in big cities for essentially my entire life.)
"start with the premise that bias isn't helpful" is, by the way, also not a measurable goal, which IMO supports what I'm saying. Knowing how to conceptualize a problem well (of which "eliminate bias" is a small subset) isn't something you can objectively measure, but it's something that will impact your objective measures down the line.
The comment about developers being autistic is particularly funny because I personally am autistic, and my wife (who you are responding to) is very much not. It was a major source of tension for us for many years.
Likewise, the emphasis on people "outside the organization" and the suggestion that she is somehow deficient on that front is laughable -- she's extremely highly regarded by customers/clients, and has been for decades, specifically for her ability to understand and solve their problems by getting them a great product whether or not they have any idea what her actual coding ability is.
And then the comment about "really wanting this to only be about code quality" is strange, since the subthread going back to kasey_junk's comment is about how "we still can’t say what good code is". Your initial response is entirely about measurable attributes of code (like code size and build time); Dove introduces "softer qualities" including "solving a valuable problem" (which is more about "quality product" than any of your metrics), and then you go back to a different set of metrics. She responded with an extended comment that specifically noted the importance of "complex impacts - on users, on business, on other programmers". Her comments have consistently been more about how good code impacts the functional product, while yours have been about measuring things about the code, but then your complaint is that she's too focused on the code and missing the point. Then you make comments about her own state of mind: what she is "wanting" to do and what you "suspect" she will do and her "complete inability" to accept certain things.
This very much feels like you just have a point you want to spike about measuring things (FWIW she does make it a point to measure things and to train her subordinates on making sure they're measuring things), and her comments (and my other one) are more excuses for you to repeat your point than actual ideas you're trying to interact with. Like you're not actually interested in engaging with the core idea that "Some of what code needs to accomplish - conceptualizing a problem well, communicating clearly - is inherently subjective, having to do with how it is received by another mind." It's just another opportunity for you to say that your set of objective measures are the only thing that matter, which, as I noted elsewhere, is itself a subjective position about which things to value.
* build time
* regression frequency
* defect quantity
* code size
* dependency quantity
* test automation coverage
* test automation execution duration
Why would these metrics be measured against human engineers when a large amount of these issues should be used to inform how to improve our systems and tooling to help devs?
For me its all about lowering regression and internal execution speed at all steps as much as possible, because I would rather do something else with my time than repeat myself on the same slow failures over and over. Getting my time back for personal use is a win for me and it allows the employer to ship a faster and more durable product.
Also, "dependency quality" just recursively moves the entire quality criteria to the next level.
> then you've basically just succumbed to Goodhart's Law
We're talking about adding more metrics that should be considered, and treating or not treating them as the ultimate goal is orthogonal to choosing the set of metrics! Product metrics don't save from goodharting
Jézus Krisztus and people are wondering why there is a job crisis underway. I had to attend my nephew's university graduation this summer and was shocked to see some 500 computer science majors, thinking where will all these people find a job? And you're talking of 5000 as a "sure thing".
Well it ain't no sure thing anymore and graduate inflation surely don't make things better.
There are ~2400 NFL players in the US, which means about 1 in 140k people.
If you figure that 10% of the people care enough about football to make an attempt, then someone who is in the 99th percentile of that group of people--someone I think many people would call undeniably good at the game, then they still have to be at least 99th percentile among those people to actually have a go of making it as a career.
There’s ~165 million males in the US, but many ex professional football players still alive and some kids who will eventually become football players. I doubt 10% of boys try out for high school or college football, but let’s say 10% make even some vague attempt.
Many people play professionally for a very short period, but arbitrarily suppose 1/7th of men are in the age range to play professionally. 165,000,000 / 2400 / 7 / 10 … So something like 1 out of 1,000 guys who make even a vague attempt end up in the NFL.
More realistically I doubt even 1% of boys ever approached football with serious intentions. Still horrible odds, except there’s a lot more athletic scholarships than openings in the NFL.
"Roughly speaking, there were 1,083,308 high school football players competing, and eventually 251 made it to pro. After simple calculation, we can get that the percentage of student-athletes going pro is approximately 0.023%."
https://u.osu.edu/groupbetaengr2367/junran-add-things-here-f...
Most players aren’t drafted. The average NFL career lasts 3.3 years. 2400 / 3.3 = 727 new players per year.
500,000 / 727 would be 1 in 687 or 0.15%.
UIUC awarded 500 ish CS degrees in 2023, so 10x less!
There's also the weird "success path" that goes from developer to manager. It's as though the end goal of learning to be a concert pianist was becoming a conductor, or perhaps a concert hall manager.
If your "success metric" is earnings, then "The force of competition at work" for knowledge work doesn't necessarily drive you to practice your Rust development "scales" every day, or to be the best Javascript dev in the team - it is probably a better use of your time to be "good enough" at your developer job, and hone your schmoozing and office politics skills to make the jump to better paid non development roles like "chief architect" of "VP in charge of {whatever}" or ultimately CTO or something.
Is it not. Wouldn't be surprised either way.
In soccer, a sport I am very familiar with on the playing side at a reasonably high level, this is not true. The median professional enjoys training and playing far less than the median spectator and median player of lesser quality would expect.
The common refrain, "If I had that opportunity, I would train like crazy," "I cannot understand how people can make that kind of money and live the dream of being a professional player and yet..." clashes with the reality of young people, like most top soccer players, who want to party, not train as much, and live a life of equal wealth but fewer obligations.
Knowledge work like programming is just much less competitive. In a graduating class of 5000 computer science majors at a good university, I’d be surprised if the majority fail to “make it” to a 100k job and be able to support themselves. Once you secure a spot in the workforce it’s pretty easy to hang onto it as an average contributor without much objective measure or comparison against your peers.
Compare to sport, at the same university maybe there are 50 spots on a sports team, and 10 good teams at the school. What percent of kids who start out playing a sport at 10 years old get to have one of those spots, and what percent of those who make the college team go on to make 100k, support their family etc playing the sport?
That competition forces rigor - if I had to compete like that to get a software job, maybe I’d be “practicing scales” on the weekend too, not just when gearing up for interviews.