This is how the new Bing Assistant works. It's also how search engines like https://you.com/ and https://www.perplexity.ai/ work - as exposed by a prompt leak attack against Perplexity a few weeks ago: https://simonwillison.net/2023/Jan/22/perplexityai/
I wrote a tutorial about one way of implementing this pattern yourself here: https://simonwillison.net/2023/Jan/13/semantic-search-answer...
It’s a small difference, perhaps, but with some significance since the retrieval and incorporation occurring outside the model has a different set of trade offs. I’m not specifically aware of any work where model architectures are being extended to perform this function directly, but I am keen to learn of such efforts.
It’s a small difference, perhaps, but with some significance since the retrieval and incorporation occurring outside the model has a different set of trade offs. I’m not specifically aware of any work where model architectures are being extended to perform this function directly, but I am keen to learn of such efforts.
It’s trained on completing the text.
If an expert write a long test and you and "in summary: " at the end, the model will complete with something approximating truth (depend on size of model, training, etc)
Humains do a similar things. We have a model in our head of the subject discussed and we can summarize, but we will forget some parts, make errors, etc. GPT is very similar.
For example, the AI doesn't have enough information about a companies process, or a regulation. It chats with an expert to fill in the gaps.
I have no understanding of AI