I don't think the gist of successfully developing software has changed that much since the early 90s, at least in consultancy: You work in iterations where the team and client develop, refine and prioritize the features of the system until the system is good enough to be considered finished. There is an initial vision and strategy that needs to be developed in the first iterations. The UP implementations describe ways on how to do this, though the agile community has come up with more creative frameworks: design sprint, product vision box, customer service map, etc. The strategy is reviewed each iteration with the re-priorization of the features. For this it's important to have revisions on the documents created on the first iterations.

The challenge as I see it is that clients usually don't buy into this and they believe that developing software is like making sandwiches or turning buttons on and off when in fact is a discovery process of abstract things. They then demand a fixed budget, fixed time and fixed milestones for a fixed scope ("You are the expert, you should know how much time this takes."). To sell the project the salespeople from software companies deliver this, those who don't are considered incompetent. In this conditions agile methodologies are prone to fail but not for the methodology, but because most of the time those sold promises are done with incomplete information or tailored to be optimistic to sell the project. Metrics from agile methodologies show from the beginning that the project won't meet the expectations and then chaos ensues: they stop collaborating with the client to hide the reality of the project, "managers" pressure the team to do more in each iteration, they stop doing the disciplines altogether, iteration and daily reviews are treated as long meetings where people commiting errors are shamed and scolded and frictions start to emerge between all persons involved. This is where I believe the whole notion of "agile sucks" comes from.