Its only linear because the process is linear as you described. There's no reason why we can't have exponential growth - produce a machine which manufactures more of itself. This machine also contains solar panels (and other appendages) to collect resources to replicate itself.

Then very soon, the earth will be overrun with such machines, and we can go interplanetary, then consume the sun to go interstellar. At least, whoever that remains after the grey goo takes everything else would go interstellar...