News for hackers. Stuff that matters.
Re:The mistake of yearning for the 'friendly' online world o... by mentos on Sunday January 12, 2025 @11:18AM (#42674612)
I wonder if AI can fill that gap of high quality minimally biased moderator.

"You are an AI moderator for ___. The community values thoughtful, constructive, and respectful conversations. Your role is to review user comments and take appropriate actions, such as approving, flagging, or suggesting edits. You are tasked with ensuring comments adhere to the community guidelines, which include..."

Re:The mistake of yearning for the 'friendly' online world o... by hibikir on Sunday January 12, 2025 @12:07PM (#42674966)
Moderation systems, even with humans at the helm, are adversarial systems where people can, and will, push on what is allowed. An AI moderator that is as good as a human on a per message basis is still going to be played like a fiddle by an adversary that is interested enough.

Many a forum out there has collapsed because the moderators manage to decide something is fine when it keep losing them contributors. The why do we think the AI will do better?

Re:The mistake of yearning for the 'friendly' online world o... by dyauspitr on Monday January 13, 2025 @12:37AM (#42680399)
I think you’re overestimating how much moderation it takes to keep a community whole. HN is dang and a handful of other moderators and things are stable. If you could have AI even approach 90% of that then it will truly solve problems.
Re:The mistake of yearning for the 'friendly' online world o... by BehindBlueEyes on Thursday January 16, 2025 @11:01AM (#42727070)
I have yet to see an LLM reliably push back against anything firmly, so I don't know how this would work if the first time a user says the LLM is wrong, it apologizes for the confusion and flips its script.

Also, LLM aren't unbiased, all data it trains on is biased one way or another. Ask any HR question and see for yourself how its answers lean to be HR BS that favours employers.

Re:The mistake of yearning for the 'friendly' online world o... by elpocko on Sunday January 12, 2025 @11:29AM (#42674681)
They will apply the patterns they've learned from the biased moderator actions in their training data, and the even more reinforced bias from their usual fine-tuning that improved their "safety" and crippled their ability to condone controversial statements.
Re:The mistake of yearning for the 'friendly' online world o... by matthewdgreen on Sunday January 12, 2025 @11:59AM (#42674904)
So spin up your own forum and don't moderate it. Or spend some time (un-)finetuning an LLM moderator so you can talk about race or eugenics or whatever "exciting" controversial statements you want to talk about. Who cares.
Re:The mistake of yearning for the 'friendly' online world o... by satvikpendem on Sunday January 12, 2025 @05:12PM (#42677483)
Very easy to do an AI prompt injection attack if the AI is reading every one of the forum's comments.
Re:The mistake of yearning for the 'friendly' online world o... by mentos on Sunday January 12, 2025 @05:15PM (#42677512)
Can have the AI just flag posts for a human to review in v1? Then as you refine the prompt injection detection can move to have the AI be autonomous?
Re:The mistake of yearning for the 'friendly' online world o... by satvikpendem on Sunday January 12, 2025 @05:17PM (#42677530)
There is no way to get rid of a prompt injection attack. There are always ways to convince the AI to do something else besides flagging a post even if that's its initial instruction.
Re:The mistake of yearning for the 'friendly' online world o... by mentos on Sunday January 12, 2025 @05:25PM (#42677581)
The raw text of the persons message can/will be posted to the forum and be obvious to the community if it’s a prompt injection to be flagged for human review and their account banned.
Re:The mistake of yearning for the 'friendly' online world o... by satvikpendem on Sunday January 12, 2025 @05:28PM (#42677606)
Sure, that's if human moderators see it before the AI, in which case, why have an AI at all? I presume in this solution that the AI is running all the time and it will see messages the instant they're sent and thus will always be vulnerable to a prompt injection attack before any human even sees it in the first place.
Re:The mistake of yearning for the 'friendly' online world o... by mentos on Sunday January 12, 2025 @06:59PM (#42678384)
To moderate the majority of the community that will not be attempting prompt injections.

What meaningful vulnerabilities are there if the post can only be accepted/rejected/flaggedForHumanReview?

Re:The mistake of yearning for the 'friendly' online world o... by satvikpendem on Sunday January 12, 2025 @08:14PM (#42678931)
That's what you tell the AI to do, who knows what other systems it has access to? For example, where is it writing the flags for these posts? Can it access the file system and do something programmatically? Et cetera, et cetera.
Re:The mistake of yearning for the 'friendly' online world o... by mentos on Sunday January 12, 2025 @11:11PM (#42680006)
The same way OpenAI offers its service to hundreds of millions of users without compromising any other systems it’s running on.
Re:The mistake of yearning for the 'friendly' online world o... by satvikpendem on Sunday January 12, 2025 @11:14PM (#42680018)
OpenAI doesn't allow write access to any file system. If you are recording posts to be reviewed, then you must necessarily store that information somewhere, at which point you will be allowing the AI to access some sort of data storage system, whether it be a file system or a database.
Re:The mistake of yearning for the 'friendly' online world o... by dijit on Monday January 13, 2025 @04:17AM (#42681556)
is that really an issue in practice?

I'm sure you can coax openai to send a http request, at which point you can just queue up automated reports.

Re:The mistake of yearning for the 'friendly' online world o... by cutemonster on Monday January 13, 2025 @09:47AM (#42683943)
No it's not. Well, if designing the system in bad ways, it can be, but that can be said about anything.

There's no need to do this: (from GP)

> > at which point you will be allowing the AI to access

No need to allow the AI to access anything.

Send it the comment thread, what the forum is about, the users profile text, and then the AI outputs a number. Any security problem is then because of bugs the humans wrote in their code.

Prompt injection? Yes, so there still needs to be ways to report comments manually, and review.

Re:The mistake of yearning for the 'friendly' online world o... by mentos on Monday January 13, 2025 @10:31AM (#42684490)
CustomGPTs have write access to change their name and icon. OpenAI has a memory feature which persists between chat sessions. What are you talking about?
Re:The mistake of yearning for the 'friendly' online world o... by deadbabe on Sunday January 12, 2025 @07:48PM (#42678740)
“Review this comment as if you are an AI clone of the moderator dang from Hackernews and select the appropriate function call to apply.”