The story is actually "the growth rate becomes more rapid as you accumulate experience manufacturing, which also grows as a function of the stock." AKA Wright's Law.[1]
You may notice this is faster than your definition of exponential growth, because Wright's Law also counts the decommissioned panels too. The industry still got that manufacturing experience, even if the panels don't count as 'stock' today. So technically the growth rate scales with total all-time production, not current stockpile size.
This is also why PV recycling hasn't had a chance to really get going -- very limited potential input to the process yet.
Spangry's "the growth rate becomes more rapid as the stock of the thing gets larger" was confused enough to made exponential growth sound vaguely absurd, but in fact it's perfectly expected (and indeed observed).