I know this is standard practice for most big companies moderating lots of content, but it has always seemed like such an insane policy to me.

Imagine if this were applied to actual laws enforced by the police. "You're under arrest but we won't tell you what law you've broken, because then other criminals might use that knowledge of the law to avoid being arrested. And by the way, a secret court has sentenced you to life imprisonment and all of your appeals have been denied."

Putting aside that law enforcement has very different risks & pressures from corporate moderation, you don't really have to imagine: US law enforcement have a tool called civil forfeiture which lets them seize assets suspected of being involved in a crime without charging the asset owners of any specific crime. The owners have to prove to the police that the assets were not involved in a crime to restore their property. The US also has FISA courts for sensitive matters, FISA hearings are secret and only involve the judge & government representatives without the presence of all relevant parties.

I'm not endorsing these US policies, but it's worth noting that even in democratic law enforcement it's accepted that the system isn't always bound by transparent policy and process. There are usually justifications for keeping some things secret and discretionary to enhance law enforcement effectiveness.

That's the claim made by Google and many other big corporations. It's plausible enough, but I haven't seen any hard evidence that it's true.

Suppose it is true that these companies can't reveal their decision making because there's so much to be gained by bad actors that game these highly centralized systems.

Then it seems like a larger number of smaller firms could be more transparent and still achieve the same effective level of security.

And we have very little incentive to not complain loudly and publicly about their practices.

If their restrictions are so trivially circumvented... then they probably already have been.

This is the epitome of security by obscurity.

Okay, but this developer isn't "everyone", and there seems to be no reason not to explain in this case.

Unless the developer decides to share on Twitter or HN or w/e, and now malicious actors know as well.

I meant that this information is not a problem to share, and that sharing information in one case does not imply sharing it in all cases.