I think the use of this term "RAG" is counterproductive and almost makes me suspicious. Because it is used ambiguously and imprecisely, usually with the main aim being to suggest a level of expertise.

But all it really means is that you put some extra information in your prompt (from somewhere) that didn't come from the user. It's basically the first thing that anyone thinks of once they get the API call working.

People don't use it to only mean vector search, despite often assuming that is what it means.

What I have been leaning towards recently is not using vectors at all, or only using it when I have an extremely close vector match. Instead I give the API one or more functions to look up information that is like an index for documentation or retrieving some info from a database. But it's specific to the application.

I think usually the non-vector stuff is more effective.