Sometimes agile is the right way to exert control back against management, if management is prone to chaos and disorder.

I took an organization that couldn't put out two releases a year to one that put out once a month, and the owner/manager complained bitterly and pushed new features into the discussion ON THE DAY OF RELEASE.

2nd level management had to constantly, repeatedly attest to the good that the tempo and structure brought to the organization and customers.

It sounds like you had a bad manager, but no methodology can fix that.

My entire experience has been: good management + talented developers = success, and bad management or untalented developers and it fails, but it really has nothing to do with the methodology in play. I've seen (and been part of) hard core agile teams burning to the ground, and teams with no methodology be wildly successful. (And vice versa)

Agile bothers me because it generally aims to commoditize developers, and yet the best places and teams I've worked on always were they opposite: it was about finding the best way for each individual to work, and designing processes around the rhythms around whatever industry they were working in, rather than assuming everything should fit into 2 week sprints.

I would argue that many people try to commoditize developers through scrum (edit: and call it Agile). Agile is just a set of principles. Which pretty much state the opposite, "Individuals and interactions over processes and tools." As the article states, scrum can work in some cases, but doesn't in others. If you keep trying to force it where it doesn't work, you aren't following the agile principles anymore. (edit: I'm probably making a "no true scotsman" augment and should've said something like "If someone keeps trying to force scrum where it isn't working they should reflect on the principles and ask themselves if they are putting processes first".)

I cringe when I hear people criticize agile for trying to "fit" everything into a two week sprint. Competent developers always chunk their work into small, comprehensible pieces, and if anything, two weeks is actually bigger than most of those chunks, not smaller. Agile never requires that everything be done in two weeks, it only requires that there is demonstrable functionality every sprint (which doesn't even have to be two weeks, it could just as easily be three, or four, or six, if that works better for the team).

this, completely.

The best places to work adapt the process to the team.

The worst enforce a process on the team because it makes managing them easier.

The number of workplaces I've been in who equate the number of hours that people are in the office with their productivity, because managers could measure attendance easily, and had no idea how to measure actual productivity...

In the environment that you describe, I imagine if there was a problem with the methodology both parties would be receptive and adapt.

Agile methodologies are weird. They seem to go directly against the manifesto, particularly "Individuals and interactions over processes and tools"[1]. Yet Scrum was created by two of the manifesto signatories.

Still, a lack of a strict methodology doesn't necessarily make it waterfall.

[1] http://agilemanifesto.org/

SCRUM predates agile. Though SCRUM has had many changes over the years, the manifesto was responding to problems of the day; that is, a giant statement of work up front, hundreds of pages of design documents, a giant build process where everyone figures out what the product really should do in engineering while product forgets about it except for occasional questions.

Then, that whole steaming pile was thrown over the fence to QA which tried to figure out heads from tails and developers started fixing the issues they already knew about but had to leave in so that they would make their deadline.

THEN came user acceptance testing, where people actually looked at the product at the first time. This is when people realized what the miscommunications 6-12 months ago, and the change request process started.

Finally, over budget and after everyone was finally exhausted and tired of fighting, it was released to users with promises of changes after the release.

From this world, SCRUM and the manifesto makes sense.

Please one element of evidence that scrum predates agile.

If you're happy to accept the Manifesto in 2001 as the start of agile, then https://www.scrum.org/about under 'Creating Scrum' states "Jeff Sutherland and I had been using Scrum for ten years prior to the meeting at Snowbird where we and others signed the Agile Manifesto"

I’ve seen waterfall work at a VERY high level where entire projects were the tasks.

Within those projects, waterfall was completely removed from the numerous tasks needed to get the project done so the developers never saw it.

Getting waterfall involved at the project level tasks day today has a lot of issues though. It can run a lot smoother if your team has a low expectation of pivoting due to new projects or production support needs.

It’s not without possibilities, but a lot depends on how far removed it is from day to day.

I believe the efficiency got "worse" because it revealed the harsh truth earlier which the Gantt chart hides... until the very end.

It's common for management to think we're making things more difficult when we're just making the difficulties more visible.

I agree about the hostility to waterfall. Waterfall put men on the moon, so it can't be such a terrible thing. I don't think there is a conflict, personally---there's no reason that you can't plan sprints in the context of an overall waterfall schedule. In the real world, there are deadlines, and any process, no matter how agile, has to recognize that fact and work within schedule constraints. I think that what agile buys you is the opportunity for near-continuous integration and feedback, so that you know fairly early when you are either going to have to apply more resources, trim functionality, or extend deadlines. Properly applied, agile is better than traditional waterfall at flushing out unknowns and potential architecture issues, because of the focus on working, tested, demonstrable code from the beginning.

The waterfall albatross is the absurd amount of up front requirement documentation in a Word file on a share drive.

Modern waterfall where you have documentation in git with references into tests is 10x better than the waterfall of old.