News for hackers. Stuff that matters.
Re:GPU-Accelerated LLM on a $100 Orange Pi by mdp2021 on Tuesday August 15, 2023 @12:11PM (#37135710)
> a local wikipedia dump

There exists (at least) a project to train and query an LLM on local documents: privateGPT - https://github.com/imartinez/privateGPT

It should provide links to the the source with the relevant content, to check the exact text:

> You'll need to wait 20-30 seconds (depending on your machine) while the LLM model consumes the prompt and prepares the answer. Once done, it will print the answer and the 4 sources it used as context from your documents

You will have noticed, in that first sentence, that it may not be practical, especially on an Orange Pi.

There are no comments on this comment yet.