The easiest way to shoot up the ranking was to avoid any work where you'd not be instantly coding. Never fix a bug where you need to spend time tracking down the issue, never take on the more complicated pieces of work, never use your time to help out junior team members. Instead focus on simple work and trivial refactoring. The statistics are vaguely interesting and I never had it used against me, but the old quote applies: "when a measure becomes a target, it ceases to be a good measure."
Yeah, I'll be honest, if I had this sort of metric forced on me, I would completely ignore everyone, not just junior devs who (quite often) ask for my help on finding the right tool to accomplish a task. I can bang out code all day, that's easy as shit.
But I think I'd quit before I'd sink to that level.
Another anecdote, I was once in a tech lead role and spent a lot of time pairing and guiding others on their development work. My commits were low but impact was high. Low commits counted against me during promotion review.
There's a thing now called GitOps, where you manage your infrastructure by tweaking things in a Git repo then Puppet/Ansible/Terraform/Saltstack/whatever runs them automatically. Easy way to get "number of commits" way, way up in a plausible way.
I have a colleague who creates a Jira for every tiny thing that for most people would be just a routine part of their job. He literally spends more time fooling around in Jira than he does actually doing his real job. But he looks great in the stats...
An acquaintance of mine told me at a networking event his company started using git stats to do performance reviews. And he pretty much anticipated how it would pretty much end mentoring, documentation and deep debugging (don't bother fixing something, rewrite the offending part of the code as much as you can and hope the bug goes away!).
I ended up poaching 5 seniors engineers from that place, and got a referral bonus for each of course.
I see this get thrown around a lot but it's not really true. If a metric can be gamed then I agree it's a flawed or poor metric, but there are definitely good metrics that make good targets as well.
Concrete example: I used to work in aircraft maintenance putting Search and Rescue aircraft into the sky to go pull people out of lakes or out of crevasses, etc. One of our KPIs/metrics was the number of hours spent in what's called the "Red" state, i.e. you have no serviceable aircraft that can fly if a callout happens, meaning the region is lacking airborne SAR assets.
There isn't really a way to (legally) game this metric. Either your aircraft is serviceable or it's not. The only way you could cheat is to just lie on your statistics and release aircraft for missions that are actually not serviceable, but that's going to bite you in the ass sooner or later, would require a conspiracy of 10+ people to lie on official airworthiness documents, and doing so is a federal crime not to mention a big ethical no-no.
Our monthly target was zero. I.e. we tried to go each month keeping at least one serviceable aircraft at all times. We only hit that target a few times while I was in that job but it was rewarding, and on months where the Red indicator was particularly large I would drill down with senior staff to determine if it was an anomaly or if there was a trend starting, and we'd address it.
And yet it's still a good measure because it's directly measuring what was our primary objective (i.e. can you put aircraft in the sky to carry out rescue missions or can you not?)
In most cases the measurements are proxies for your "share" of the job. In these case they are almost always bad. In all my years in tech I've not seen a way of directly measuring what I do.
Exactly!
Team metrics can be useful. It is important to know if, say, a team is missing deliverables more than another team. Then you can potentially track down _systemic reasons_.
On an individual level? Can't do it, there's not enough context.
On a pure "lines per code" metric, the most important members on my team would have an abysmal metric. But if they didn't do their jobs, the productivity for everyone else would drop enormously.
Indeed.
Is there a clause I'm unaware of in the saying I quoted that stipulates a target/metric must be an individual one only?
Most of these processes are designed to compensate for managers who don't know what the hell they're doing and shouldn't be in charge of anyone.
I have seen git commits change over time because someone started to look at stats. You absolutely get what you benchmark for.
Nobody cares for effectiveness above a certain company size. Even if the company crashes the managers will still get their bonuses and golden parachutes...
So you end up with work being done to fit the metrics, not necessarily work that is of highest need or more impactful to the organization.
I have seen plenty of bad metrics lead to managers no understanding why their dept's appear to be "hitting all the numbers" but also no performing in the way they desire at the same time
Proving that the initial problem can be reduced to a people problem is useless.
To be more specific about metrics, something that measures what your users care about, or is beneficial to your company is actually useful. Sometimes the most important metric is a zero or one, which is did you ship? Measuring commit frequency seems intrinsically useless.
If you are covering up for something (no staff) than you will look bad. Push the problem up. Make the people above you look bad instead of taking every bullet.
Obligatory reference to (generalised) Goodhart's Law:
"When a measure becomes a target, it ceases to be a good measure."
Yeah, GP's comment was completely off base. O365 is user-hostile tracking-ware by design. git is just a tool with incredibly high granularity. We don't say scalpels suck because psychopaths use them to torture people.
That sounds awful. Sorry that happened to you.
I've been an engineering manager, for a time, and I used Gitprime to track stats.
It was never used for performance reviews or in making promotion decisions or anything.
What it was used for was to manage upwards and demonstrate to the upper management how decisions affect the team, to indicate when we needed to spend on hiring and had room to onboard new people, etc.
I don't see metrics as "surveillance," unless they're used that way. They can be used for good. But I think as you and others have pointed out -- it requires context and transparency: you can't just rely on metrics for reporting on a person.
Tech managers: you need to bring your people up, develop their skills, and pay attention! Metrics are useful but only relationships bring context.
(I eventually returned to engineering as I didn't find I enjoyed management much)
you have login times to see if they are late, chat metadata to see time to response on dm, idle status to see if keyboard / mouse went inactive. meetings attended, page views on the intranet, etc. if your a full stack o365 shop, then you can essentially log any and all activity for most employees, and therefore target kpi's that you want. even it admins to a degree will have these logged. powershell sessions opened, commands executed against the stack, every single click 8n the web gui in o365, every single action in azure. they are all logged and assessable, I used the same data for security monitoring before I moved to a Google shop
Once the company got bought, my new boss in charge of all developers across all divisions asked me to not mention that again so he doesn't have to do that. Once I moved on, it sounds like ParentCo took a more active role in the development and hopefully they stopped that.
But the tool doesn't force managers to use it, and the absence of a surveillance mechanism, someone who seeking an excuse will invent a metric for the purpose.
The only cure I've found for toxic culture is to remove someone from it. Usually me.
Definitely that.
Git is the first thing I set up when I start a new side project, every job aside from one which let me set it up, have used version control. I will say Plastic SCM sucks and no one should use it though. One of my side projects ran into massive issues with my contractor due to no one, including myself understanding how plastic works. GitHub for life
I've worked (thankfully not as an employee) for a well known BANK which did not have any kind of version control whatsoever. None. Not even folders with dates.
"Where's the latest release?"
"Oh, it's in John Doe's flash drive over there"
I am not kidding.
After a lot of pestering by my company (which was working on a consulting role), they spent big bucks on some unknown version control system used by hardly anyone else. It followed the old Source Safe's "lock" mechanism. Except you had to use their own software and navigate to each file in question (no searching). Imagine how amazing it was to work on a java project and sloooowly open each folder in the reverse DNS package convention until you got to the file you wanted. Very slowly, because all actions happened on the server.
Later on they were bragging on how they were able to create a branch in less than a week. Merging was a day long affair with all code contributors present.
EDIT: still, that was over a decade ago. I didn't know there were companies involved with software in any capacity that still don't have version control in 2020.
"As an (I like to think) good boss, I might use Git statistics to steer interventions with under-performers. If they're committing zillions of lines of code, it's maybe an efficiency or expectations problem. If they never push commits before noon it's maybe a motivation problem."
Though he backtracks a bit regarding his own personal approach:
"But it's always just one data point, and almost always one that colors a response to an issue that exists independently. I'd never determine that someone was an under-performer based on their Git commit history."
The only places I'm aware of that don't use it are game companies, which tend to use source control systems like https://www.plasticscm.com/games specialized in better handling of very large binary blob asset files like textures, movies, audio clips, etc.; and Google, which has the monopoly-scale dev budget to be able to suffer heavily from "not invented here syndrome," so they built their own proprietary in-house SCM.
I can't imagine working somewhere without git, I think the last time I worked somewhere that used something else was a mix of git and cvs back in maybe 2010ish?
Any version control?
Because if so ... wow. That's bad, I thought that sort of practice went out in the 90s.
Most places I've worked in the last decade use git. A big bank used private instances of bitbucket or stash or something. Another fintech was on github, a smaller company had gitlab or something set up. All git.
Outside of git, one of the tech behemoths had several of its own systems going, clearcase and something in rational team concert. Even the tiniest, worst-run places I've been to in the last 15 years or so have at least used svn or something, and going back to the beginning of my career, 20 years ago, there's always been PVCS or SCCS or something...
Honestly I would find it hard to claim that any serious sort of software development at all was going on if there wasn't some sort of vcs!
Used SVN previously for private stuff and I think it is still spread in many companies. I liked it, just all the small folders are a PITA and git is faster for larger repositories. It had far better tools than git in my opinion, TortoiseSVN. Heavy, but practical.
I would always use git now, even if only locally.
It's also well supported out of the box by Azure Devops (aka VSO aka TFS) so I would expect to see it in more traditional, non tech companies as well. The idea that it's not enterprise ready is ridiculous.
Going by completely subjective metrics is pretty unfair, everyone has some amount of bias.
When there is a team member where you feel something is off from a performance perspective and they should do the same things as other colleges and there is a hard difference between them, it helps you that your gut feeling is not that off.
Any company can create bad metrics if they want. The only solution is to either play the game, or change companies.
That said, I fail to see how effective employee monitoring would be via Office 365. I don't know who would be evaluated on number of emails sent per day, unless they were some sort of support personnel. And if so, they are already being monitored by other means. Any other type of evaluation sounds absurd, ex. number of Word docs opened or created, number of spreadsheets opened, number of presentations created, etc.