I can only speak for tilde.fun, but it's helpful to not have any users currently (^^) and disable outside communication apart from ~/html/ where our users can put static files into.

Most tilde sites don't have public mail servers and only federate mail between them and other tilde servers. See also https://tilde.team/.

Do you use a VPS on AWS or something like that, or an actual physical box?

It's currently a VPS rented from Strato.de, one of the biggest German hosting providers.

Since I'm paying out of my own pocket I currently don't want to afford a colocated server, even though I realise that'd be cooler and possibly more secure.

I'll try to have a detailed cost overview online somewhere soon-ish.

I admin a smaller pubnix/tilde, for me I tie down email, inbound connections, some strict resource limits per user, and keep a close eye on anything running. I'm sure if I start getting more users it'll become more time consuming but with a decent logging and sensible security practices you could probably negate the vast majority of bad actors.

I'm a volunteer admin for tilde.town and I also run trash.town

We mostly monitor resource usage, and built in a way to ban users from our django-based administration app. We have begun screening users more before allowing them to sign up, asking them things about what they want to use the town for.