News for hackers. Stuff that matters.
Re:What Makes a Senior Software Developer? by thebent on Sunday December 11, 2016 @11:48PM (#13155617)
"Every measure of a good developer is qualitative. That makes them all subjective."

^ This post scratches the surface on a timely and relevant question many teams face, but the author seems to throw hands up and say everything is subjective.

Our work suggests otherwise: https://blog.gitprime.com/check-in-frequency-and-codebase-im...

Re:What Makes a Senior Software Developer? by zaptheimpaler on Monday December 12, 2016 @06:18AM (#13157265)
Im currently working on performance optimization. Often my job is days (even weeks) of running tests and reading metrics, followed by a 5 line change one day that significantly speeds things up. This is one reason why metrics cant hope to capture everything. Maybe its not all subjective but a lot of it is - especially for more senior people, its about how much contribution they have to the overall business goals rather than any data you might find in the codebase.

Ofcourse there are people who do this much better/faster than me, but the nature of the job still means the actual code output is pretty low.

I can imagine many other functions where code output is super high (adding lots of text of any kind - blogs, templates, new library/config) or super low. I don't think code output correlates well to productivity.

Re:What Makes a Senior Software Developer? by Bartweiss on Monday December 12, 2016 @10:32AM (#13158713)
I have a longstanding suspicion that these issues are why there are so many different 'measurement' structures for programming, which don't ever converge to a few winners. I totally believe the GitPrime result is accurate, but my first thought was "I wonder which teams this doesn't apply to?"

'Programmers' is a coherent grouping in terms of "people who write code as a job", but it doesn't generalize well to "people whose daily work is similar". At the extremes, it's a bit like calling a typist, a copy editor, and a poet all "writers" because they produce text. That's a useful grouping when you're designing Microsoft Word, but it's a lot less useful when you're judging everyone on "lines written today". (I worry that there's a value implication in those three careers, and it's not intended; I just wanted vastly different daily workflows.)

And so the GitPrime result seems useful to know about, but there are clear cases like yours where its inapplicable. I suspect it would also run into issues with things like embedded or critical-failure code - NASA's commit patterns might have nothing in common with the industry standard. Even in my largely standard work, I realize that of my last two projects, one had ~50 commits and the other had ~3. The 3-commit one was a tech-debt project with lots of reading and documentation work, and I'm skeptical the two are at all comparable.

Even within similar work, I imagine things like branching strategies and test coverage alter commit frequency. For the near future, we'll probably continue to see the value of a given assessment stay company-and-role specific.

Re:What Makes a Senior Software Developer? by hermitdev on Monday December 12, 2016 @10:47PM (#13164271)
My entirely subject opinion.

Those that self-identify as programmers may or may not be current HS or undergrad students. They often know a single language moderately well, maybe a second. Though they know the language, they often don't know what is available to them in the language of choice's standard library, so they often fall victim to re-inventing the wheel.

A developer has a few seasons under their belt. They may know several languages, and even have a fair grasp upon what the standard library of those various languages offer and so less likely to reinvent the wheel (although they still sometimes do). They're more likely to have a larger toolbox and decent grasp on when it's reasonable to use a certain tool (language or framework to tackle a new problem).

An Engineer is more likely to involve tools to solve problems. Got a performance problem? Going to profile instead of running by gut feel. If your company can afford it, vTune is a great tool here. Engineers are more adapt and comfortable using tools such as gdb, valgrind and strace on Linux or say Windbg on Windows. Bonus: they usually don't need to refer to the manuals to specify the correct commands or interpret the results. This doesn't come from accident or pure reading - you have to have real world practice and experience to know when and which tool is appropriate for the problem at hand.

The Senior Engineer knows all of the Engineer as I've briefly outlined above, but also takes the time to teach/mentor/coach the juniors around him or her when they see room for improvement in the junior. And this should not be in the form of "you're doing it wrong, do this instead", but there should be a "why" component in the "you're doing it wrong".

Re:What Makes a Senior Software Developer? by twic on Tuesday December 13, 2016 @02:48AM (#13165253)
Your description of senior engineer fits me reasonably well - eight years experience, have had 'senior' in my title for the last three jobs, routinely teach the people I'm working with - but I identify as a programmer, not an engineer, because there is no such thing as software engineering, and if by some chance there is and I've missed it, I'm certainly not doing it.
Re:What Makes a Senior Software Developer? by ktRolster on Monday December 12, 2016 @07:02AM (#13157437)
If you use a metric like that to affect my pay, I will happily maximize the metric to ensure I get paid as much as possible.

As a bonus for you, I will also try to write good code.

Re:What Makes a Senior Software Developer? by wccrawford on Monday December 12, 2016 @11:16AM (#13159139)
So true! I have yet to see a metric that wasn't gamed somehow, even by people who are generally honest and upright otherwise.

If you base someone's livelihood on a number, expect them to attempt to maximize that number, probably at the expense of other numbers that you may or may not be considering.

Re:What Makes a Senior Software Developer? by marssaxman on Monday December 12, 2016 @07:48PM (#13163529)
If you use a metric like that to affect my pay, I will go find a different job, where I am treated with respect.
Re:What Makes a Senior Software Developer? by touristtam on Monday December 12, 2016 @08:15AM (#13157802)
Gamification at its finest. I copmpletely agree with you. :)
Re:What Makes a Senior Software Developer? by thebent on Monday December 12, 2016 @07:11PM (#13163346)
“to affect my pay” ← totally agree with you here.

This is a big part of why we think engineering metrics are a tough nut to crack: there’s a fair bit of research that suggests tying KPIs to compensation is a huge anti-pattern for certain types of work (chiefly those involving lots of novel problem solving).

Highly recommend Daniel Pink’s “A Whole New Mind” on this topic; it’s pretty fascinating stuff.

Re:What Makes a Senior Software Developer? by thebent on Monday December 12, 2016 @10:22PM (#13164175)
Highly recommend Daniel Pink’s “A Whole New Mind” on this topic

Correction, the Pink title I was referring to was "Drive: The Surprising Truth About What Motivates Us"

Pink does a great job unpacking research that demonstrates how this type of 'affect my pay' dynamic backfires for engineering.

Re:What Makes a Senior Software Developer? by ronald_raygun on Monday December 12, 2016 @01:03PM (#13160181)
Reminds me of a good essay - "You are what you measure"
Re:What Makes a Senior Software Developer? by debaserab2 on Monday December 12, 2016 @02:09AM (#13156220)
I agree with the overall message of the post but not the analysis behind it. It depends completely on the codebase and where you as an engineer are assigned to work in that codebase. There's so many potential ways for bias to be introduced into that kind of analysis (e.g., what if I am a junior dev assigned to copy edits to the static sections of the website?). There's a qualitative perspective to commits that's going completely ignored in this post.
Re:What Makes a Senior Software Developer? by marktangotango on Monday December 12, 2016 @08:50AM (#13157988)
This ties into what I was talking about above with respect to "heavy algorithmic stuff". Most developers can work with intricate legacy code by adding an if here and there to handle special cases. Very few can recognize that it's encoding a state machine with state transition function Y, and rewrite it. Even then, very few have enough 'attention to detail' to do the rewrite, and NOT lose any legacy functionality. This is an example of what I consider to be 'hard stuff'.
Re:What Makes a Senior Software Developer? by Bahamut on Monday December 12, 2016 @11:15AM (#13159119)
This is why I like when people contribute to major open source projects - chances are good they will be exposed to some of these hard problems, which can help them solve future ones.
Re:What Makes a Senior Software Developer? by thebent on Monday December 12, 2016 @11:06AM (#13159034)
@debaserab2 100% agree, well said.
Re:What Makes a Senior Software Developer? by Eiriksmal on Monday December 12, 2016 @12:14AM (#13155720)
Engineer happily using GitPrime here. After sitting in on a Skype session we had with your team once upon a time, I took your earlier (pre-Impact) observations on the correlation between commit volume and overall performance to heart and consciously focused on chunking my work more. It will come as no surprise to you that this strategy's helped me stay consistently in the upper right of the 2x2 chart. Thanks, Ben!
Re:What Makes a Senior Software Developer? by Xcelerate on Monday December 12, 2016 @07:27AM (#13157562)
> I took your earlier observations on the correlation between commit volume and overall performance

What does this mean? The best developers tend to have the most commits?

Re:What Makes a Senior Software Developer? by jiaweihli on Monday December 12, 2016 @11:59AM (#13159565)
I can believe that there's _some_ correlation.

From what I've seen, a high frequency of commits means either:

  - Has very little understanding and churns out code one day and removes it the next.
  - Thinks a lot about their code, and divides them into digestible, independent chunks.
Re:What Makes a Senior Software Developer? by ____nope on Monday December 12, 2016 @02:57PM (#13161353)
So it correlates either to a lazy developer or a really good developer? That isn't much of a correlation.
Re:What Makes a Senior Software Developer? by jiaweihli on Wednesday December 14, 2016 @01:47PM (#13178548)
It's some correlation, and it's more useful than the full spectrum.
Re:What Makes a Senior Software Developer? by thebent on Monday December 12, 2016 @12:37AM (#13155836)
Ah nice, @Eiriksmal — 'once upon a time…' and 'pre-impact', sounds like this was a while ago now :) Encouraging to hear that this has been helpful for you.
Re:What Makes a Senior Software Developer? by Bartweiss on Monday December 12, 2016 @10:44AM (#13158824)
This is really interesting, thank you.

It's one of the only clear-metric breakdowns that actually makes sense in light of my programming experiences. I'm hesitant about the idea of using it as a management tool (easy to game, and high commit count or even 'high impact' isn't proof of quality). And, as some other people noted, it's quite role-specific - a performance engineer's commit count won't be comparable to a line engineer.

But as a personal insight, or a mentoring tip? This seems excellent. I know I want to write good code, I'm not going to game my personal metrics, so a clear guideline like "break up code more" is a great thing to have.

Re:What Makes a Senior Software Developer? by thebent on Monday December 12, 2016 @11:05AM (#13159023)
@Bartweiss thanks for the feedback. The thing about “gaming the system”, is that it presumes an adversarial relationship between engineers and non-engineers. That’s pretty unfortunate, and it’s our view that a good portion of this is due to non-engineers not really understanding what happens software development.

But yeah, my co-founder has a pretty interesting take on gaming the system.

TL;DR: So, can you game the system? Absolutely. The point is to game the system: https://intercom.help/gitprime/general/cant-you-just-game-th...

Re:What Makes a Senior Software Developer? by Bartweiss on Monday December 12, 2016 @01:26PM (#13160420)
Thanks, that was a great quick read.

Adversarial work does seem like it's largely a sign of communication breakdown. I think 'gaming' isn't always as adversarial as it seems, though. I was considering it in terms of Goodhart's Law, where even with good intentions, representative metrics become less representative as you organize around them. The "active days" entry in your link seems like an example; if everyone pushes most days for the sake of pushing, then active days isn't feedback on programmer skill.

Of course, your link touches on that. "Everyone pushes every day" is a great outcome, even if the metric is no longer clear. The common trend is to pick good metrics, then struggle to keep them relevant. I really like the alternative of just setting metrics that will be good after people optimize.

Re:What Makes a Senior Software Developer? by hulahoof on Monday December 12, 2016 @08:13PM (#13163669)
The other side of this is it is likely if the engineers are optimising for a metric that is important enough to be tied to compensation, the companies' non-engineers are likely to place a high value on that metric too, exacerbating the issue.

This is a situation where I feel gaming the system is not an inherently adversarial relationship.

Re:What Makes a Senior Software Developer? by gaastonsr on Monday December 12, 2016 @12:21AM (#13155760)
Love this, in my experience also, good programmers make a lot of small commits. While less experienced people do ocasional commits regardless if they are big or small.

Something your chart doesn't show is people who make a lot of commits/work but the quality is subpar. They behave as prolific programmers but still need to improve.

Re:What Makes a Senior Software Developer? by thebent on Monday December 12, 2016 @12:34AM (#13155824)
Thanks for checking it out @gaastonsr — yeah, you make a great point. we've figured out some ways to account for that in the software itself.
Re:What Makes a Senior Software Developer? by Quarrelsome on Monday December 12, 2016 @04:23AM (#13156778)
surely this assessment introduces false positives? Someone that implements a poorly specified feature exceptionally well suffers rework, not because their code is poor but due to outside factors. Someone who cherry picks easy items is reflected as prolific where a perfectionist might just be given more difficult problems.

Idk, I worry that people will interpret this backwards and rather than having characteristics of a good developer ape those characteristics to appear as a good developer.

Re:What Makes a Senior Software Developer? by Bartweiss on Monday December 12, 2016 @10:39AM (#13158775)
I think Goodhart's Law is in effect here.

As a management metric, I would find this concerning - perverse incentives and false positives all over the place. But as a personal observation, I'm going to take it to heart.

This (like most other metrics) seems most useful from a starting point of "while doing my daily work and seeking to write good code, I will apply this". That doesn't make a good tool for deciding who's talented, but I'm still going to remember it.

Re:What Makes a Senior Software Developer? by nine_k on Monday December 12, 2016 @09:40AM (#13158285)
Whatever you measure, when people know it, will affect how people act. People will try to move the needle in the direction they see profitable, even unconsciously.
Re:What Makes a Senior Software Developer? by shandor on Monday December 12, 2016 @01:27AM (#13156060)
That's actually fascinating. Is the formula for the mentioned "developer impact" a hush-hush trade secret, or can you go into more detail on how you calculate it? Would be really interested in hearing more!
Re:What Makes a Senior Software Developer? by thebent on Monday December 12, 2016 @10:58AM (#13158952)
@shandor — yeah, there's some secret sauce happening in that impact metric, but this post breaks it down so you get a sense of how it's calculated and how teams are using it: https://blog.gitprime.com/impact-a-better-way-to-measure-cod...
Re:What Makes a Senior Software Developer? by SeanDav on Monday December 12, 2016 @04:53AM (#13156915)
The danger of developing a qualitative metric for work/job performance, is you now have a recipe to game the system for anyone that is so inclined. Of course, if this metric is used as an indicator for promotion and raises, nearly everyone will be inclined to game the system.

It might not even be as overt as that. It is a well known phenomenon that what you measure, you improve.

So as soon as you have a quantitative metric for work performance, almost by definition, it can become problematic to use, unless you are very careful how it is used.

This is not to say that finding a qualitative method is useless, far from it. It can lead to valuable insights and improve overall quality of the team/business.

Re:What Makes a Senior Software Developer? by SeanDav on Monday December 12, 2016 @08:37AM (#13157914)
Oops messed up my qualitative and quantitative. All references should have been "quantitative". This is what happens before the first coffee!
Re:What Makes a Senior Software Developer? by cbcoutinho on Monday December 12, 2016 @04:23AM (#13156781)
You guys maintain an excellent blog - I would really like to keep up-to-date, but I don't want emails sent to me.

Can you guys supply an RSS feed link? If you have one, I wasn't able to find it.

Re:What Makes a Senior Software Developer? by thebent on Monday December 12, 2016 @11:00AM (#13158969)
@cbcoutinho appreciate that. i think this should work: https://blog.gitprime.com/rss.xml
Re:What Makes a Senior Software Developer? by mkalygin on Monday December 12, 2016 @03:50AM (#13156635)
Nice article indeed, thanks for sharing. I'm currently struggle to go from perfectionist with low frequency to high frequency.

I've also noticed that experienced programmers do their commits more frequently and they do the right job - they spend less time on unimportant details.

It also depends on the codebase. A programmer who works on similar codebases for 10 years is more likely to have a high impact than the one who work less and consider these codebases rather new. While their approach and skill set may be similar.

Re:What Makes a Senior Software Developer? by pmarreck on Thursday December 15, 2016 @10:49AM (#13185346)
Quantitative information can only ever be used to inform qualitative valuation. It should not be absolutely linked (in other words, you should not make hire/fire decisions solely based on metrics of any sort). Your work is definitely interesting, though.

Note that every metric can be gamed, by the way.

Re:What Makes a Senior Software Developer? by bananarepdev on Monday December 12, 2016 @10:25AM (#13158653)
So you are not measuring the value delivered by the developer or his team to the business, but rather the amount of work done.
Re:What Makes a Senior Software Developer? by JamesBarney on Monday December 12, 2016 @04:46PM (#13162211)
What was the dataset you guys used? Was it open-sourced or closed source?