You're assuming those who have left share a uniform ideology. Many who left wanted far stricter moderation than Musk. Some wanted less. Some wanted more tolerance, some wanted less. There's also a vast difference in moderation of a single centralised system and a network of thousands of instances - you can be opposed to the moderation policies of a centralised network and still think it's fine for those policies to exist somewhere as long as you don't need to deal with them.

I just want to remind you that Mastodon has existed for about a decade, way longer than any of the Twitter stuff has been going on. Culturally it is the same as it always has been. Why would I as a server administrator have to change because new people are interested in our platform? If you join my server and don't like the experience, feel free to move to a different server or to start your own.