Natural resources, Earth's capacity to sustain our growth, and later on, speed of light (unless we break the FTL, there's only so fast we can expand our economy). Every exponential growth in physical world (as opposed to mathematical theory) has a limit at some point.