Actuaries, quant finance (it's not dead yet), crypto-related programming, logistics... there are lots of non-academic jobs (most of which pay better than all but the best academic appointments). It may be the case that giving up four years of earnings is not worth it from a purely pecuniary standpoint, but I seriously doubt that there is an overabundance of math PhDs given the jobs they can take.
The private sector is, of course, much less visible than academia.
A math PhD in an actuarial setting is out of place, like a CD grad working at Geek Squad (a hypothetical Geek Squad with pay comparable to programming).
While having the degree helps you get the job ("omfg you have a PhD you must be so smart!"), it isn't necessary. The number of jobs where PhD level mathematical knowledge is necessary is actually quite small. Producing more PhDs will probably not be as helpful jmtame seems to think.
Plus, a reasonable fraction of the academic positions are purely teaching.
http://www.ams.org/employment/2007Survey-DG.pdf
I know physics is similar.