Re:Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI...
by aznumeric on Thursday September 05, 2024 @06:35PM
(#41460973)
One way people keep costs down when using OpenAI with an offline RAG system is by limiting the number of text snippets sent to the API. Instead of sending the whole database, they'll typically retrieve only the top 10 (or so) most relevant snippets from the vector database and just send those to OpenAI for processing. This significantly reduces the amount of data being processed and billed by OpenAI.
[ Parent
| on HN ]
There are no comments on this comment yet.