What implicitly stops 0.1% of people from controlling 51% of the network? You realize how much of global fiat wealth is currently in the hands of the 0.1%, right?

The current system at least theoretically has democratic (or otherwise) control over who the 0.1% in power are. In a system like BTC or ETH it's just 'who has the most compute?' or in proof of stake, 'who's the richest?'

If any group controlled >50% of the network and also was willing to use that power in a way that would diminish trust in that same network then they'd very quickly end up with 100% of something worthless.

This seems to imply that everyone else's stake in the network would become worthless too, though. What stops a particularly motivated billionaire from buying 51% compute for a month or so to destroy a cryptocurrency for good? They waste that money but any competitor of theirs (in business, etc) who had wealth tied up in that currency can kiss it goodbye. If a business was built around that currency it's dead now.

The 'diminish trust in the network' threat really only discourages an existing good actor (invested in the network) from turning into a bad actor.

If I've been mining a cryptocurrency for years and I decide I want to move on to mining something else, what stops me from using 51% to destroy the currency before I move on to mining something else? If I moved my currency holdings out (or have been liquidating them regularly, as any sensible miner might), what's to lose?

They still own a bunch of hardware, though, which can be repurposed for other networks. If you're a big miner, and something on whattomine.com seems attractive, you'd join the network, mine it, unload via exchanges and leave for greener pastures once the ROI is not that attractive.

That implies there is a viable alternative people can go to. I'd be more concerned that a group with >50% would go after alternatives/competitors.