What you propose works in a small organization - which is the type of place where scrum was created and works well. In a large organization there would be too many people trying to make conflicting changes for that to work. Any individual change might be good, but they conflict and nobody can track what you are supposed to do now. That is why large organizations are so hard to change.

Agile focusing on people is a good thing. People are number one, but agile has always acknowledged the roll of processes.