In an odd sequence of events, I've sorta got a weird tiny following in China due to having a vintage camera stall in an antiques market here in Wales that gets shared on Xiaohongshu (https://en.wikipedia.org/wiki/Xiaohongshu) sometimes when I have Chinese customers. So I'd feel bad blocking the entire country myself, though thankfully I'm not being inundated with traffic from there yet - the most egregious bots I've caught so far are actually ones from Meta which ignore robots.txt.
Oh trust me, we felt really really bad blocking the whole country, and it took weeks of attempts avoiding it.
Meta bots are also problematic, as they seem to be quite buggy. But the volume was nowhere near the hundreds of bots from China :/
Yeah Meta's bots are well and truly stuck in the blackhole on this site at the moment. I have no idea what they're supposed to be doing, but in case they are using them for AI work I've just added a token multipler based on subnet totals so they can have some more fun. I might come up with some other fun response types for them too.
I am surprised POW Anubis doesn't work for you. Are these bots bypassing POW somehow?
POW never worked. The only bots its stops are the simple ones that don't support java script or cookies at all.
Around June/July we got a 10x boost out of nowhere, and it started affecting performance for users, increased hosting costs, and random bursts would bring the website down.
We spent some time trying out solutions, from Cloudflare and Anubis to AWS, but it ended up affecting real users, and we got complaints on Reddit from day one, pretty much. At the same time, it got a bit expensive having a WAF.
In the end the only solution that worked was blocking every single Chinese IP.
Traffic instantly got down to 10x less, same as it was before. Same conversion percentage as before.