Even GPT-3.5-turbo-16K isn't good enough for most retrieval augmented generation tasks.

Locally ran LLMs are far worse.

I don't like it either, but for now, if you want good RAG, you have to use GPT-4