I am sorry that was your experience, and not because of "agile" — businesses which leave expertise to experts usually excel IME. The one gotcha is that you want to have good questions asked along the way (by PMs, business people...), and not necessarily have the "last word", when smart engineers will realize sooner when they are on the wrong path. This combination of talent is rarely achieved without specifically recruiting for it.

To back this up (since we are presenting anecdotal evidence), I've been at different software engineering teams at engineering companies which did provide this approach to work, and I was at pretend-agile shops too. As a software engineer of more than 20 years.