News for hackers. Stuff that matters.

A comment on OpenAI ChatGPT: Optimizing Language Models for Dialogue. Parent comment

Re:OpenAI ChatGPT: Optimizing Language Models for Dialogue by gamegoblin on Wednesday November 30, 2022 @09:10PM (#33811360)
Seemed vaguely like how the brain does it. You think a thought, and somehow your brain conjures memories that are semantically related to that thought. That sounds a lot like a nearest neighbors algorithm on a language model embedding vector to me.
[ Parent | on HN ]
Re:OpenAI ChatGPT: Optimizing Language Models for Dialogue by visarga on Thursday December 01, 2022 @04:35AM (#33814178)
There is a paper from DeepMind

RETRO - Improving language models by retrieving from trillions of tokens (2021)

> With a 2 trillion token database, our Retrieval-Enhanced Transformer (Retro) obtains comparable performance to GPT-3 and Jurassic-1 on the Pile, despite using 25× fewer parameters.

https://www.deepmind.com/publications/improving-language-mod...

[ Parent | on HN ]