News for hackers. Stuff that matters.
Re:Teaching GPT-3 to reverse words by yunyu on Monday May 16, 2022 @06:20PM (#31403814)
You can update the "graph edges" with content longer than the window by fine tuning: https://beta.openai.com/docs/guides/fine-tuning
Re:Teaching GPT-3 to reverse words by f38zf5vdt on Monday May 16, 2022 @06:41PM (#31403974)
Yes, training the model is where it learns, not in prompts. Prompting might be considered meta-learning but it will always need a reference point given to it from its training data, and beyond the prompt the original model is never altered.
Re:Teaching GPT-3 to reverse words by sdenton4 on Tuesday May 17, 2022 @01:52AM (#31406475)
Eh, one could argue that this is similar to the short term/long term memory divide in humans. We tend to suck at new things until we sleep on it and update our weights...