In a recent previous thread here on HN I was roundly criticised for saying that due-date driven management is not effective, but that’s what scrum has become, and todays thread is exactly what I was thinking about.
My experience in a few businesses is that Scrum as it is generally practiced in industry has been an absolute disaster for productivity. Kanban is awesome because it doesn’t force us to spend hours and hours every week on ceremonies which don’t change the outcome one whit.
In my own business, after 2+ years of scrum - including hiring a CSM - I tossed out our entire process, and replaced it with backlog management and kanban boards. The only thing we kept was the (short) daily standup.
Productivity went through the roof for a number of reasons, not least of which was that because we could now spend the ceremony time working on actual code.
The killer is that if you try to tell people this, they think you’re inexperienced or did scrum wrong or you don’t know what you’re talking about. But to me Scrum is a two year black hole of stagnant product development that I will never get back.
Do you have any recommendations for scenarios where there are due dates? I work in a field that is largely focused around dates that aren't really negotiable; the school year ends and starts and school budgeting decisions have to be made at a certain time, so if we're rolling out certain features, they really do need to be available in, e.g., August when school starts.
I've been using a backlog-driven, priority-based approach in my own practice for the last 5+ years, which has been extremely successful at delivering products, but remarkably less successful politically. Non-technical management seems to have come to the conclusion that we ought to be able to accurately estimate the time it will take to build stuff, despite decades of evidence to the contrary. Loads of people argue that this is the "only way" but the processes commonly implemented are more about making management feel good than about actually getting work done. Unfortunately, I've found that making management feel good is often more important than actually doing the work.
All of that said, what I suggest is very boring: instead of spending the (significant) time estimating the effort required to implement each task, instead you should spend the time working with the client on prioritising the features, and internally spend time prioritising the decomposed tasks. What features/tasks absolutely must be ready by the deadline, and what can they live without? Can some features be shipped "incomplete" from your perspective? For example, can you deliver a veneer over a manual process on launch day, and automate the process later? If, upon inspection, a feature turns out to be extremely complex and time consuming, can it be re-prioritised? Can you spend time working on reducing the total amount of actual effort that's on the "critical path" to the deadline? Obviously some things just need to be done, but when you drill down you often find that a lot of time is spend on features and tasks that are not critical (by whatever definition you choose to use).
The idea is simply to make sure that the important stuff gets done first, so that you are in a position to launch as soon as possible. You can and should continue to work on the project after the launch day. This approach has many well-known carry-over benefits like the ability to start user and automated testing very early. Unfortunately, as mentioned, this can be very tricky to pull off politically, which I think is why it's not more common.
Once you get to this point then reporting upwards should be about features delivered, features we're working on now, and features that have not yet started. Somewhere in that list is a red line that says "we can launch now". You can directly measure velocity in terms of how many features are left until you are in the green.
I want to be clear that I think rough-order-of-magnitude estimates on work-in-progress and future work are perfectly fine and reasonable for internal use. What I am against is the use of deadlines and dates to manage the project as a whole. The goal is to deploy features, and that will take as long as it takes, and no amount of stomping of feet is going to change that, assuming you have a good team.
Ultimately what everyone wants is for the project to launch successfully. Software is very complex, and history (and theory) suggests that it is impossible to estimate accurately. The more time we spend on estimating, the less time we spend on working, and the more likely we are to miss our deadlines.
One final note. If you use JIRA, the backlog sucks (or at least it did when I abandoned it a few years ago). You need something that you can manipulate, that you can create different views, and that can collect information that helps you manage and prioritise it, since the backlog is orders of magnitude larger than the in-progress work and in my opinion is the actual problem that needs to be solved. I've personally used ProductBoard for many years but AirTable and even a spreadsheet is good.
I am keen to hear criticism and other ideas. My key thing is that saying "this feature should be finished by THIS DATE" is an absolute killer of productivity because of the amount of effort required to determine "THIS DATE" and the fact that the date is always wrong anyway.
This isn't even remotely true. Kanban establishes specific measures, including throughput, that, combined with statistical modeling, specifically allow for forecasting to be done.
Do you really think the car industry, where Kanban originates, doesn't care how many cars they're going to produce in a quarter, or whether they're ahead, behind, or on track on hitting their targets?
Come on.
The idea that "forecasting" and "agile" are somehow at odds with one another is a gross misunderstanding of both.
Edit: As an aside, that article's use of Google as somehow the paragon of software development superiority really dates it. Google is notorious, at this point, for being an unreliable software development organization. They fail to listen to their users by building features people don't want, canceling projects that vocal users do want, and shipping incredibly buggy products (Android 12, anyone?) All of this is, at least in part, caused by many of the problems the blog post highlights but claims have been resolved, including a lack of strategic cohesiveness and issues with building and maintaining staffing for projects that are no longer sexy.
That's interesting to me as I hadn't heard that about Kanban before. Do you have a particular reference in mind that you think describes this well?
Then you can measure throughput (number of stories per unit of time) and cycle time (amount of time a story takes from start to finish).
Using those measures you can forecast, with monte carlo simulation as a common method. Here's a book on the topic:
https://www.amazon.ca/Forecasting-Simulating-Software-Develo...
There's plenty of other resources about this if you bust out your google-fu.
CPUs can only execute, what the instructions tell them to do—like a production line—and not solve / create something (new).
Kanban doesn't "fix" this -- it just changes the frame through which deadline-buzzards get to look at it.
It’s certainly possible to estimate to a rough-order-of-magnitude but the problem is that the ROM quickly becomes a target, and then a commitment.
I have a couple of positions that I take.
First, the old adage still applies: “better, faster, cheaper - choose two”. If you set a deadline then you have to be willing to drop features or quality, because Brooks already demonstrated you can’t add resources after you start.
Second, if you do need to set a deadline then prioritise it, stick to it and celebrate it. Too many times deadlines are used as a whip, and my or my team have wiped themselves out to meet it, only to be told it wasn’t really needed. This is insanely demoralising and strongly affects productivity in the long term.
My personal (and unpopular) preference is to prioritise completion of stand-alone features rather than building to deadlines, which means you can release a product at almost any time with fewer, but better quality features. This approach has a demonstrable positive impact on the ROI but because it’s more complex and provides less short term certainty, it’s frowned upon.
And it’s well established that a rewrite is dangerous. The second system effect suggests that rewrites will always take much longer.
If it reduces friction (as suggested) for both sides and everyone spends less time managing ceremonies as a result, then its a net win.
BTW, one indication that Scrum is bad is that nobody is doing its analogue (syncing everybody once a day?) in computing. Sane project management practices should be universal, and should also work for operating system schedulers.
In Agile, "points" are just an abstraction around work to try to normalize it in the same way that every burger+fries combo should be roughly equivalent.
It essentially claims that "good agile" is Kanban, and Kanban is essentially abolishing of any forecasting in favor of actually doing the tasks, like the CPUs do when executing programs.