Do we have any examples to draw from operating at that scale? Due to the Musk purchase, we have a rare public disclosure of a major tech giant's internal workings where Twitter claims (https://www.theguardian.com/technology/2022/jul/07/twitter-s...) it suspends half a million spam accounts a day. It's trivial to file an appeal, so (because the attackers are by no means disincentivized from appealing) that's probably on the order of 250,000 appeals a day if we assume even half the banned won't bother (and I think that assumption is low, since a reasonable system should probably make it trivial to file an appeal, therefore people will). How does one reasonably handle that? To scale up to thousands of humans handling it, you'd need strict rules to even approximate getting the same result across everyone, and if strict rules worked you could code them into the auto-ban system.
One solution might actually be "Stop having 'get every human being on the service' as a goal," but then sucks for the people who can't use the thing.
"We charge $100 to have a human look at this case, if the suspension was incorrect we reinstate the account and your money. If not, we tell you what actions you can take that would un-violate our policies so you could be re-instated."
Spammers doing selenium powered spam would never pay for that, but the real folks who are not tight with large audiences would have an option.
I can't imagine Twitter could get away with this without it being interpreted by the regulatory bodies of various countries as a protection racket. Which is unfortunate, because I think there's potential here.
Well yeah, it's trivial to file an appeal if you just have to click a button, but obviously that's not what I was talking about. Have they tried literally anything else?
Solve difficult CAPTCHAs, send a scan of your passport, pay a small amount of money (e.g. $1), put a large amount of money in escrow like someone else suggested, allow other well-known accounts to vouch for you, etc. etc. etc.
There's a ton of things you could do. It seems like they've given up without trying any of them.