News for hackers. Stuff that matters.
Re:Clanging by ganzuul on Wednesday December 27, 2023 @06:50AM (#38781112)
Does RAG alleviate this?
Re:Clanging by hhh on Wednesday December 27, 2023 @09:01AM (#38781961)
No, not really. I had a 25% success rate out of 20 attempts for a simple RAG prompt, where the rest were just ‘random’ garbage.

I wouldn’t expect it to alleviate this.

Re:Clanging by ryanklee on Wednesday December 27, 2023 @10:27AM (#38782883)
Why would it?
Re:Clanging by ganzuul on Wednesday December 27, 2023 @10:56AM (#38783185)
Maybe root it in reality.
Re:Clanging by ryanklee on Wednesday December 27, 2023 @12:54PM (#38784509)
RAG just puts more info into a prompt. It wouldn't effect the underlying behaviors in any way a non-RAG supported prompt would.
Re:Clanging by ganzuul on Thursday December 28, 2023 @11:29AM (#38795153)
You can get it to be more creative with prompting.
Re:Clanging by ryanklee on Friday December 29, 2023 @02:52PM (#38809433)
I'm aware of that. My point is that if you have fundamental incoherence in the model behaviors, you're not going to solve it through prompting. To get performance gains through prompting, you first need model behaviors that are able to be improved through prompting. RAG and prompt engineering work on the margins, not the main.