The labor is a small fixed cost. Depending on what is done with the labor the leverage is potentially infinite (eg. many internet business that earn a million or more per year for every employee they have hired).

Growth can happen by getting the same labor to do something more valuable (maybe building houses instead of chairs?)

Labour is people, which need food and water.

I don't see the point of your statement. Economic growth can happen even if the population remains constant (or even shrinks but that would be a headwind to overall economic growth and at that point it might be more instructive to look at gdp per capita).

No need for these fancy thought experiments - you're just talking about efficiency.

Either way there's a finite amount of resources. Even if we were 99.999% efficient, eventually, it runs out.