The problem with zero growth is that we would also need to have zero population growth. There are two ways of achieving this: the nice one, where we get everyone out of poverty and people naturally stop having so many kids because they're not poor any more, or the nasty one where we impose limits on how many kids people can have. The nice solution would definitely require lots of economic growth to lift all of us out of poverty (without reducing the living standards of the people currently having no kids to the point that they start having kids again). The nasty solution has only been tried by China so far, wasn't that successful, and has lead to all sorts of demographic problems for them.
We do have a problem with income inequality which should be a lot easier to fix than stopping growth.
But how far in the future is this? Our light cone is finite, but it has a vast amount of energy and matter. Maybe we don't get much father than the solar system, but that's still a lot more resources than the Earth. Just the sun itself is an enormous ball of energy we're barely making use of. Sure, a Type 3 civilization might find it difficult to grow much beyond it's galaxy, but it would take a while to get to that level. We don't really know where civilization might end up in the long term.
So whether or not one calls for zero growth, we should plan for an eventual situation of zero growth and structure our society so this is a good outcome.