Long term (~50 year) average stock market returns on reasonable indexes (like S&P500) is 10% per year.

The only real challenge is a portfolio mix and withdrawal strategy that minimizes the damage from withdrawing during downturns. That's where the 4% rule becomes relevant.