The problem that I would see is that since wealth tends to be passed on to future generations, without growth it would be difficult for those who currently are not wealthy to ever become wealthy.

This is maybe a similar point, but it also seems to me that the without growth we would need to come up with some alternative to 401ks in order to allow people to build wealth.