Do you mean that number are to coarse to indicate a specific problem?
Of course there is the problem of gaming the numbers. But we do think that compared to many other ways of measuring productivity (for example the number of issues solved) this is relatively robust against manipulation. Getting something out sooner is better most of the time.
But thanks for the thoughts and I hope you try it out soon.
My own view is that robustness to manipulation can't come from metric design, it can only come from culture.
If metrics are tied to reward or punishment, then no matter how clever they are, they will be gamed.
Then there's Time to Customer Value, which is the time it takes before a customer using the feature turns a buck on it.
> there is a risk of starting to optimize for the numbers
There absolutely is. You can only use your own judgment of the balance of risk between flying blind and becoming obsessed with instrumentation.
https://en.wikipedia.org/wiki/Slang_terms_for_money#United_S...
It's probably not precisely measurable, but it's a great thought exercise. It prompts us to think about the entire process from noticing an idea to a customer deciding to buy.
And Time to Customer's Customer Value -- I forgot that TTC^2V was the alternative name -- is even better. It makes us think about the journey from us noticing a need, developing it, releasing, customer installing, their team using it, releasing to their customers, who decide it's worth paying for.
When you turn that into a loop, you get something very like the Gitlab master plan. We call it the "Circle of Code", Onsi Fakhouri talked about it early this year: https://youtu.be/7APZD0me1nU?t=23m6s
Personally, I have mixed feelings about feature flags. They're the worst thing after feature branches, which are the worst thing after feature flags.
That said, I see with this demo how these metrics could be useful to track, well, when reviews are taking too much time (needs more people? better repartition of reviewers?) – or when maintenance tasks are slowing new features (needs better tests? more maintainers?). I guess I'll have to try this out :)