News for hackers. Stuff that matters.
Re:A guidance language for controlling LLMs by andai on Tuesday May 16, 2023 @04:43PM (#35967488)
Have you used the OpenAI embeddings AI? It is used to find closely related pieces of text. You could split the target text into sentences or even words and run it through that. That'll be 5x cheaper (per token) than gpt-3.5-turbo and might be faster too, especially if you submit each word in parallel (asynchronously! Ask GPT for the code). The rate limits are per-token.

Not sure if it's suitable for your use-case on its own, but it could at least work as a pre-filtering step if your costs are high.

(The asynchronous speedup trick works for gpt-3 too of course.)

Re:A guidance language for controlling LLMs by jweir on Tuesday May 16, 2023 @04:58PM (#35967670)
I have not yet played with embedding. It is on my list though. Fortunately for my current purposes 3.5-turbo is fast enough and quite affordable.