What you describe looks how agile was supposed to be. What the op described looks like a perversion of agile perpetrated by managers who either don't understand why agile was born, or do understand it, but only care about increasing their power instead of getting things done.

I know. When I first read the XP book by Kent Beck it was a revelation. That was the first time I saw a methodology that I felt worked and agreed with my own experience. Planning everything out in advance is hopeless.

There are two great quotes that I think capture this:

“A complex system that works is invariably found to have evolved from a simple system that worked.” John Gall

“Enlightened trial and error outperforms the planning of flawless intellects.” David Kelley

It is really sad to read about how often the agile ideas have been misused!

How about "a team of skilled developers is likely to be successful with any software development approach; an inexperienced or untalented team is doomed regardless of the chosen methodology"

After 20 years that's been my experience

Yes. But unfortunately not every team can be made entirely of rockstars. So the purpose of a methodology is to increase the probability of success for a moderately talented team.

Between you and what OP is saying, I think there is this something thing else also. Core software development activity (UI, databases, messaging, web servers, configuring legacy systems, testing, release and delivery, etc) is helped by people sitting together on an agile track. However for some software development this also means architecting the product itself, an aspect that is crucial to a company's survival too. In a company building a demand management product, I need to learn and understand the market, consumers, sales processes, our internal systems that manged these before drafting the demand models, fine tune the methodology, etc before anyone really build the software product/services that reach the end-user. And there is a cycling back and forth between the two activities as we move ahead to get things right. I had no product manager or analyst who could really help with the product itself that the company wants. This latter part is where where I have a very hard time sitting in an open space with 10 people talking on the phone or getting pulled into frequent meetings.

I have no doubt you're right.

The thing is, you can have development standards that require small iterations that are immediately deployable and this are immediately tested and deployed, without pulling all the overhead of some PMs interpretation of scrum.

I often find the power is generally with the wrong people and they promote the things that matter to them at the detriment of everything else.

I don't personally believe this is by design, it's just people in $foo management position tend to have the soft skills to make their job easier, when many of the developers often don't. And managers, who tend to control budgets and define departments, tend to give those departments to more managers and there's not a lot of people in the room telling them that a software person should probably run a software team.

Have you asked people who work at Ericsson today? From what I have heard it is hard to think that things are better today despite moving to things like agile.

I work at Ericsson today. Unfortunately, the environment is very much like the one described by the OP: open plan office, noise, chit-chat, interruptions, daily stand-ups (which are mostly a waste of time), scrum meetings which take place far too often, retrospectives which turn into blamestorm sessions. All of this makes the office a rather unpleasant place to be. Which is a shame, because whenever I get to actually sit down and get some work done, I tend to enjoy it.

You'd think I could cope with this by working from home, but that option is limited, too.