Many a forum out there has collapsed because the moderators manage to decide something is fine when it keep losing them contributors. The why do we think the AI will do better?
Also, LLM aren't unbiased, all data it trains on is biased one way or another. Ask any HR question and see for yourself how its answers lean to be HR BS that favours employers.
What meaningful vulnerabilities are there if the post can only be accepted/rejected/flaggedForHumanReview?
I'm sure you can coax openai to send a http request, at which point you can just queue up automated reports.
There's no need to do this: (from GP)
> > at which point you will be allowing the AI to access
No need to allow the AI to access anything.
Send it the comment thread, what the forum is about, the users profile text, and then the AI outputs a number. Any security problem is then because of bugs the humans wrote in their code.
Prompt injection? Yes, so there still needs to be ways to report comments manually, and review.
"You are an AI moderator for ___. The community values thoughtful, constructive, and respectful conversations. Your role is to review user comments and take appropriate actions, such as approving, flagging, or suggesting edits. You are tasked with ensuring comments adhere to the community guidelines, which include..."