The issue is that creating objective measure of contribution value is just unfeasible if at all possible. Not to mention amount of work required which is exactly the issue.

But mostly, maintainers want to just focus on their work and not be bothered by corporations and their developers trying to game the system just to prop up their position.

> The issue is that creating objective measure of contribution value is just unfeasible if at all possible.

Could you give an example of a non trivial patch which value would be unfeasible to assess objectively? I am not familiar enough with the maintenance process to understand how hard it is to measure objectively the quality of a patch.

It is not about any single patch, you can't assess value of any patch objectively.

For starters there isn't even an objectively established definition of value. You can't objectively measure something that only has subjective definitions.

In a free market you could say that the price could be proxy for the value (ie how much somebody can pay for it should at least theoretically correspond to its value for that person). But how you do that for an open source project?

It might be feasible to assess a single contribution, but not feasible to assess all of them. To make this KPI work, maintainers would have to publish an objective quality rating for each accepted commit. That's an enormous amount of work, and impossible to get right anyway since the value of a change might not be immediately obvious, or might be negated by a future change, etc.

The advice to companies using this KPI would be: let your own dev managers evaluate the quality of the commits of their engineers, and rate them on quality as well as quantity, then submit. Engineers will quickly stop creating busywork when their managers get wise to it.

Yes, so either accept the status quo or create positive change.

Given that it’s annoying enough that we have articles and comments on it, perhaps investing maintainers time into developing metrics that work for the maintainers isn’t a terrible idea.

What for? Metrics for metrics sake are a waste of time. The solution is to avoid wasting time on metrics, which is precisely what happened here.

Sometimes, throwing trash into the trash bin is positive change. You don't need to put new trash in as a replacement.

Doesn't work. "When a measure becomes a target, it ceases to be a good measure."

You can totally build something that is a decent metric, but as soon as you create incentives to "game it" (optimize for the metric not the actual goal you're trying to measure), it will be gamed, and creating metrics that are resistant to that is nearly impossible in most cases.

Edit: Nvm, I misunderstood the initial problem and my solution doesn't address it.

Why not just keep commits as a metric, then estimate the quality of those commits by sampling. If the organization appears to be gaming the stats then flag it as such.

I tend to look on commit count as a negative when reviewing open source projects. TeX is probably the highest quality open source project and it's has had maybe 15 commits since I was born. Bash has had about 15 commits in the last five years. Then there's everyone else who's playing to win the Github game with 500 stars and 5,000 commits. That level of activity might impress normal people but it doesn't impress me.

> I tend to look on commit count as a negative when reviewing open source projects.

I do agree that tracking commit counts isn't very useful but tracking commit frequency, in my opinion, is quite useful. I'm obviously biased but I use commit frequency to tell me how fast a project is moving and what is the investment.

Take the following for example:

https://public-001.gitsense.com/insights/github/repos?p=impa...

by breaking down how frequently contributors commit on a daily basis, you can get a very good sense of investment and speed. In the case of the vscode project, Microsoft is investing a lot of resources into vscode and it is evolving at an extremely fast pace.

Software metrics in general isn't the issue, it's the lack of context around them that is the issue.

Disclaimer: I'm the creator of the tool that I'm linking to

Because that would be a huge time sink for the maintainers as well. They’re the only ones qualified to evaluate quality and they have better things to do with the time it would take. This problem is created by these companies and it’s their problem to solve.

I like this idea, but the gamification / rules / meta around it sounds like it would be even more work. And like all metrics it would quickly become a target and... Goodhart's law.

This is one of those ideas that I think should totally be a thing, and yet I think wouldn't work and thus shouldn't be a thing.

I’m not knowledgeable enough to define what good work looks like but, simply, whenever maintainers see good work, give that user a golden star.

People will work for the carrot, so give carrots for the work maintainers want to see.

Then you get them into trying to get their own people into maintainer positions to let the golden stars flow. Maybe also be more hesitant to give out this starts to the "wrong" people from other companies. Not like there already is enough politic bullshit in open source.