I think gp got it the wrong way round. Single front rings came in because front mechs were always a reliability weak spot as unlike the rear they must shift chain under tension. Also if you ditch them you can run narrow wide chain rings which hold the chain better on rough ground, and you get more clearance under the bottom bracket.

After we realized we didn't need 30 gears, new suspension designs also became possible.

You're right that rear mechs frequently die though.

The rear derailleur is protected by the presence of the wheel. The bigger danger is getting the front gear caught on things and tipping over on logs or sharp drops. Moving to a smaller front gear gives more clearance for the more likely snags

This is an interesting question. I just went out to look at two bikes I've got, both Santa Cruz, one of them a 26" Nomad from ten years ago, and the other a new Hightower 29er.

I'm pretty sure the derailleur clearance is actually lower on the old 26" bike! The geometry of the derailleur has changed, and when it's on the biggest rear sprocket, the old Nomad derailleur's center of mass is below, lower than the large sprocket. On the new 29er, the derailleur is mostly above the bottom of the largest sprocket. It seems like derailleurs have improved in order to allow the larger sprockets that single front chainrings call for.

The idler (lower pulley) on the 29" is almost certainly further from the axle than on the 26", however the larger wheel means that clearance from the ground is about identical.

I don't think I've ever hit my derailleur on either bike for lack of clearance -- even though I have caught my pedal on a rock or branch multiple times. I have hit the derailleur during a crash, but mainly because the bike was flying around and smacking into things. I'm guessing because the derailleur is behind and inside the path of a pedaling foot, it's mostly protected.

Aside from the front chainring being more exposed than the rear derailleur, the rear derailleur also tucks away quite dramatically when you upshift. Nobody is in the biggest gear on the way down.

It does, but at the same time as chainrings dropped from 3 to 2 to 1, the wheels grew from 26 to 27.5 to 29 inches (for the most sold bikes), so I imagine in the end perhaps the difference is not so big?