News for hackers. Stuff that matters.
Re:Stanford Alpaca, and the acceleration of on-device LLM de... by int_19h on Tuesday March 14, 2023 @01:58AM (#35147906)
This is the wrong way to go about it. Instead of trying to force that informaiton into the model via training, you need to set it up such that it "thinks out loud" as it answers the question, generating intermediate queries against your data sources in the process.

Take a look at this chat log from Bing. Note that the user only sees the stuff that is marked #message and #suggestions.

https://www.make-safe-ai.com/is-bing-chat-safe/Prompts_Conve...

Then take a look at https://langchain.readthedocs.io/en/latest/, specifically the Document Loaders and the Indexes sections to begin with.

Re:Stanford Alpaca, and the acceleration of on-device LLM de... by a_bonobo on Tuesday March 14, 2023 @03:07AM (#35148253)
I've been playing a lot with the langchain Document Loaders - their examples make the results look more impressive than they are, or perhaps that's because our documents are 'special' (heavy in taxonomy/biology). I've been priming it with 'if you don't know, say that you don't know, don't invent anything' and it usually ends up saying 'I don't know' to questions it can clearly answer. But yeah, working on that part!