Not necessarily.

You don't need to move the entire industry off-planet physically to reach a state where there is little industry on-planet and lots of industry off-planet. Tools can be made in space with materials on-hand up there. With those, you can get access to more materials (asteroid mining), and build more tools, more industry, etc.

The only real barrier right now is efficient transportation. Ion drives look poised to solve that problem.

But the effort of creating that infrastructure burns energy and creates pollution that wouldn't exist otherwise.

Also true. I think we're not going to get a good handle on what is achievable and what is optimal for a few decades at least. I have no idea what space industries might look like in say 100 years if the cost of access to space is less than 1% of what it is now, so I don't think it's a decision anyone is in a position to take right now. But short term, by which I mean the rest of my lifetime of 35+ years if I'm lucky, it's looking interesting.