Re:Running large language models like ChatGPT on a single GPU
by simonw on Monday February 20, 2023 @02:59PM
(#34872169)
That's not necessary. Look at how a Bing works: it's a LLM which can trigger searches, and then gets fed the search results back to it as part of the prompt.
I wrote about one way to implement that pattern here: https://simonwillison.net/2023/Jan/13/semantic-search-answer...