-- Value individuals and interactions over processes and tools
-- Value working software over comprehensive documentation
-- Value customer collaboration over contract negotiation
-- Value responding to change over following a plan
Most of the problem described in the article -and everywhere- is precisely that "the theory is good but everybody is doing it wrong".
But that doesn't really help, because it overlooks the fact that the origin of all this is just a pile of generic, hard-to-argue small-time guru ruminations. I mean, it is not necessarily bad advice: "These are valuable things and we think that in particular these ones are more valuable than those other ones". Okay. Probably. They do look good. Let's say they are. Now what? Oh, that's left as an exercise to the reader.
It's funny. Just yesterday I almost had an argument over one of these things. I heard Karen say to the guy on my left that "Agile means less documentation", so I raised my eyebrow. "Agile means less defined documentation, more open, because documentation has no value at all" - she added. I was about to point out that that is not at all what it means, but then she went further and told him that what he needed to do was to "work as a team and become more involved in collaboration". That, right there, is the problem: pretending a bunch of cool sounding expressions can be a substitute for actual things like writing down what they wanted the guy to do.
There's value in enumerating important principles for consistently pursuing quality work product. That's why the Agile Manifesto has value. The manifesto is just a manifesto, a statement of intent and principle, and not a methodology in and of itself. One doesn't need a published, monetized methodology to engage in agile software development. One simply needs a methodology -- whether unique to one's project, or widely known -- that reinforces those principles to engage in agile software development.
The biggest problem with agile software development as a concept is that it, like any other elaboration of any principles of quality work, is that most people will fail to see the forest for the trees, will marry themselves to some well-marketed thing that pretends to be about principles, and will end up largely doing the same dumb crap they've always done.
Sometimes just documenting what the thing is supposed to do and why would solve so many issues, but since people have gotten hooked on capital-A Agile so much has gone by the wayside, or been delegated to non-technical roles.
-value trust over plotting and scheming
-value open communication over hoarding information and deception
-value disarmament over preparing for battle
While we value the things on the left, we value not getting steamrolled by people who practice the things on the right even more.
No, that's not the problem with the five values and twelve principles.
In fact, the problem is the desire to have a single, divinely ordained methodology (which both the first value and twelfth principle reject, and that rejection is arguably the central basis of Agile.)
OTOH, while it is good that the Manifesto and Principles don't ordain a methodology a it's bad that they are written in a way which makes it harder to extract actionable direction than is necessary.
And we're going to do agile! It will be great! How are we going to do so? According to this rigidly-defined process, that's how.
But I think the bigger problem is that often developers and management (at all levels) give very little thought into "delivering value" and a lot of thought into "doing whatever they want".
I can't fault individuals (workers, management, or executives) for trying to do what's best for their careers rather than what's best for the company. As a manager, I think management's primary objectives are to set up their reports for success and to figure out how to align their reports' incentives with the company's incentives in the best possible way.
Management, I have seen, often does a terrible job 1) actually knowing what "value" is, 2) figuring out how to measure that they're actually delivering value, and 3) figuring out how to align delivering value with the worker's own incentives.
If you can't do these things, you have NO chance to do agile effectively. It's easy to go through the motions of Agile, but you will gain nothing if you can't define and measure value (and hardly any companies can -- it's not easy!)
But as you pointed out, even if you can accomplish the prerequisites, it's still INCREDIBLY difficult to attribute an engineer to value. You can execute effectively, but be on a project that turned out to be useless. Making the button green might be easy, but because of previous technical debt, making it blue might be really hard, etc...
* Often the financials of a company are "need to know" or obfuscated (Y-Axis unannotated)... * The financials are regarded as irrelevant to engineering (by management, finance AND engineering) * The estimates of where the value lies is often just a spreadsheet sucked out of the thumb of somebody in marketing who already has a eye on his next job. * Nobody _ever_ goes back and closes the loop and evaluate the value estimations and the value estimation process, so the company never learns to do better. * Let a drooling slob of an engineer talk to a customer? You're joking Right?
And yet engineer is all about tradeoffs, we make several per day under the illusion we know which are wise....
That sounds like a cultural problem. From my experience, tools do not solve cultural problems. Leadership changes cultural problems.
I'm saying that it seems to be a problem at the vast majority of companies of all sizes.
When I worked in teams following the agile process, we were rigid.
Agile shouldn't be a process, mgt see it as such. It becomes a process. Clever engineers hate poorly thought out processes. Agile as a process is a contradiction in itself.
Agile is no process. Let the team find their ways and iterate as often as possible, along with a few other proven principles.
Points becomes estimates for managers, they will see it as such and build huge waterfall projects documents to track everthing, and they will pressure teams to deliver shitty software when they feel the stories are sand bagged.
Strange world we live in.
When I worked in teams following the agile process, we were rigid.
"The Agile that can be named is not the true Agile."[0]
I'm not being entirely facetious.
The software architecture is falling apart, the tech debt overshadows the actual debt of a small country, our subject matter expert quit the company over being micromanaged, and the clients are abandoning our buggy software for our competitors? What went wrong, we closed all of those JIRA tickets, surely that must be good for something.
I’ll add this — To a large extent, many management systems are about reducing the failure modes — but I don’t see why a system must downplay creativity and deep thinking about the core business engine (often the software).
Yup that's exactly it. Or to put it more delicately (than simply a "power grab"): a façade that servers various ulterior purposes -- from making their company seem hip and modern (to the uninformed) to simply being able to say "finally we have some structure!" even though it's very debatable whether new "structure" really brings the benefits that it claims to (and evidently brings many negatives on board with it).
But ultimately - a façade.
Of course, just as with engineers, there are good and bad managers. But it seems pointless to always complain about management being the ones screwing it up all the time.
Any attempt to persuade him otherwise just results in him pointing his raygun to his watch. Either he's ignorant, can't or won't understand, or simply doesn't care.
Software management in 2019 is still very much like this. Management has had 40 years to solve this problem, and it looks like it won't be solved in the next 40 either
It’s as if we were complaining that elected officials get the crap for mismanaging the country. Well, that’s the point of them doing their job. Our goal is not to all become managers (though some do, exactly for these reasons), it shouldn’t be a ”why don’t you do it ?” rhetoric.
I just mean that the "metrics" put out by Agile give the illusion of something useful to managers. It's totally understandable that folks in positions of authority would attempt to make use of them. But I think the illusion of objectivity is more damaging to teams than the acceptance of uncertainty.
It's where I think Jira and other do-everything tools that try to be useful to both ICs and management fall down hard, every time I've seen them used. Management wants to see their burndown charts and such. They want projections out months on minimal data. They want to see the performance of people, not teams. Points become BS because everyone knows they're being watched. Totals are missed the first few sprints (of course they are, you're still calibrating WTF a point means for this team and this project!) and management gets pissy, exerts more control, implies ICs are failing at their job, project goes downhill from there. But look, we're Doing Agile! Such professionals we are.
I’d love it if developers would be willing to collaborate using a dedicated project management tool and leave the bug tracker for bugs, but the chances of that happening voluntarily are nil. You gotta go where everyone else already is.
What'd that look like, that's different from what they do now? Trying to imagine what developer input to that would look like. I think a lot of devs already feel like existing PM tools they use for sprint tracking and such are worse than if they pushed more planning into git or somewhere otherwise closer to the code, so getting buy-in might not be so hard if devs got to be a little less heavy in a bunch of not-actually-that-useful-to-them tools on a day to day basis.
I think 1/4 of the challenge is what tool to use and 3/4 is convincing and motivating people to be reliable and forthcoming with accurate information. A good project manager demonstrates he or she can add value, unblock things, escalate and communicate appropriately, and not just be the Project Police.
By using things like story points and other metrics intended to be internal to the team, developers have a lot of incentive to not lie on their estimates. On the other hand, management has incentive to push things to be finished quicker.
By letting management having access to those metrics, they're automatically at a disadvantage when negotiating timelines and priorities.
Almost every time I bring this up somewhere, someone pops up to say that metrics are important, key to fair and impersonal management. Without it, everything would suck, and managers couldn't do anything. The problem with that thinking is two-fold:
1. It ignores the fact that, as soon as there's a metric that determines someone's rewards and punishments in a management system (whether it's governmental regulation, corporate middle-management, anarcho-communistic self-management, or anything else), the system of measurement will get gamed, and the most overtly "successful" will end up being the most cleverly sociopathic about it.
2. There is an alternative to strictly quantitative evaluation. The alternative is qualitative evaluation. This requires good judgement, intimate knowledge of the problem domain, good intentions, and the ability to consider context, among other things. Sadly, this is incompatible with bureaucracy, which means you need to either carve a bureaucracy-free zone out of the larger organization to manage a team well or stay out of medium-to-large corporations (and out of small businesses that ape the behaviors of large bureaucracies).
Managers who insist on doing everything quantitatively (and sometimes they're forced to do so by their managers) will not get good results. "Good" is qualitative, not quantitative. Equating "good" with metrics just gives you results that look "good" on paper, but really aren't good.
Probably the biggest benefit of a small organization is the ability to eschew managerial bureaucracy in practice.
They do: engineering-led companies are often dynamic success stories, then when they get big enough traditional management works down from the top to lower levels and the firm stagnates.
They're useless if you're giving management direct access to team planning & communication tools, of course.
The places I've had it succeed greatly have all been new development. We consistently scored stories. Product owners could then had a budget equal to the team's velocity to purchase from the backlog into the next sprint. What it really looked like was they got to arrange the order that the backlog would be picked up and how much would get done in a sprint was an estimate, but with consistent implementation it became eerily accurate over time. It has been my observation that it takes about 6 sprints to calibrate -- where the team scores consistently and the variances from sprints cancel out.
Another point of emphasis is that velocity is a team score. Every failing effort I've been involved with used it to score individuals which inevitably ends up in scoring the stories higher so that an individual gets more points. It also results in cherry-picking easy points and ignoring tasks and bug fixes which may result in zero. It also (like bug fixes) results in everything being assigned points such that points become a proxy for time. If you're developing a new system and you have a bug fix, the team has already claimed the points for that bug fix. You should not make more points for the bug fix, your velocity should actually go down, giving a more accurate estimate in the future of how much new system is actually produced and not a measure of how much a programmer has worked.
So, measurements should absolutely be available to management. But they shouldn't be used to impact future scoring.
I can't throw it all on management either. People who should know better will accept 2 days worth of points for conducting training or attending a workshop.
Largely, though, I would say it's a simple matter of not understanding how Agile works at all. It has just become, "you score points for doing stuff" (in proportion to time, but it's not time, but it is).
[0] 'Measuring and Managing Performance in Organizations', Robert Austin, Dorset House, 1996.
I guess upper management could enforce the separation on a level lower management could not override, but I would assume such cases are rare.
We work in a field where inherently there are unknows, and measuring the unknown is not gonna be that useful, however we turn it.
Eventually, things changed, but for a while there it was awesome. Things got done well, in reasonable amounts of time, with a lot of attention to quality work.
I think the beginning of the end was probably around the time when the company (a software development consultancy) started using fine-grained hourly measures of time pre-sold to clients. This started happening around the time investors in certain clients got more involved in trying to dictate company policy, and I'm pretty sure it was all part of an attempt by investors to ensure there were "better metrics" for evaluating their investments.
You can't measure something without it directly affecting the measured thing's state. In this case, it started having detrimental effects on almost everything. Previously happy and productive devs started burning out, mediocre devs became new management favorites, more bugs got into production, projects suffered under less and less reasonable goals (including at one point my task effectively being to solve one of the Hard Problems of software dev -- cache invalidation), and so on.
In the US, management culture seems to have a monomaniacal focus on metrics. (I'd say "simplistic metrics", but the way they use them guarantees they're always simplistic.) This is driven, in part, by bureaucratization, which is in turn driven by a bunch of other things (mostly stemming from politics, directly or indirectly) in our business culture. Many managers would complain that without metrics they can't evaluate team productivity, individual value to the organization, and other important stuff, and they're right. The solution is not to double down on those metrics obsessions, though; it's to replace those managers with managers who are capable of effective, qualitative evaluation, instead of the almost purely quantitative approach that holds the high ground.
One nice thing about this is that the metric is visible. Then you can optimise your performance against it. Is this necessarily productive, or in the real interests of the company? Maybe, maybe not. But I look at this as a kind of a favour: if your boss is complaining "velocity is too low" well, at least you know what you're being measured by. Put some extra points on those stories. Undercommit. Make your boss happy and get them off your back. You'll be doing the same work either way.
There is, I admit, an element of it in the sprint planning, but that is only meant to be a tool to figure out how much work can be put into the next sprint. Agile does not care what your absolute value of velocity is.
If however, some misinformed PM is obsessing over velocity then I think that is a very different issues and not really related to the essence of agile.
It wasn't a conclusion it was a summation of what is happening when agile ideas are misused. I've seen it used that way too, it never ends way because nobody can measure a programmer's productivity/effectiveness.
I suspect that many devotees of agile would speak at this point about getting management buy-in, or reforming the wider organization, but at this point "agile" just becomes shorthand for "reform your entire organization from top to bottom so that your management doesn't suck".
I think the issue with Agile is that software is a field in which is difficult to measure productivity easily. Then Agile comes along and offers the appearance of making it easy to score and and measure team and individual effectiveness. Sure it's "supposed" to be for the team to measure their own effectiveness and empower people to work more productively, but for folks in management roles, the opportunity to 'score' engineers based on Agile metrics is too attractive.
Once it becomes just a way to figure out who's not moving fast enough, or "failing to meet their commitments", of course people will hate it. It gets changed from a tool into a stick.