News for hackers. Stuff that matters.

A comment on Google Books Is Indexing AI-Generated Garbage. Parent comment

Re:Google Books Is Indexing AI-Generated Garbage by candiodari on Friday April 05, 2024 @12:06PM (#39944086)
Not true. There's plenty of easy ways to update the weights if you want to do that.

You could for example DPO based on another LLM's evalution of how a conversation is going, or you could use whether the conversation keeps going.

[ Parent | on HN ]
Re:Google Books Is Indexing AI-Generated Garbage by staticman2 on Friday April 05, 2024 @02:44PM (#39945655)
>>>"You could for example DPO based on another LLM's evalution of how a conversation is going,"

You can update the weights based on a single point of data (response was bad) but you probably can't usefully update a model that way.

[ Parent | on HN ]
Re:Google Books Is Indexing AI-Generated Garbage by candiodari on Friday April 05, 2024 @04:54PM (#39947080)
I can't seem to find anyone actually usefully trying this. Do you know of any data?
[ Parent | on HN ]
Re:Google Books Is Indexing AI-Generated Garbage by staticman2 on Saturday April 06, 2024 @04:33PM (#39955256)
No but Google (for example) recommends fine-tuning based on at least 500 examples of question response pairs. DPO as far as I know requires a good and bad example. It's not a technique based on just saying "ai response is bad".
[ Parent | on HN ]