You're assuming those who have left share a uniform ideology. Many who left wanted far stricter moderation than Musk. Some wanted less. Some wanted more tolerance, some wanted less. There's also a vast difference in moderation of a single centralised system and a network of thousands of instances - you can be opposed to the moderation policies of a centralised network and still think it's fine for those policies to exist somewhere as long as you don't need to deal with them.