Re:LibraryBox
by qgin on Tuesday March 05, 2024 @12:34PM
(#39606643)
A future version of this might just be a local, offline LLM
[ Parent
| on HN ]
I am a techno-skeptic most of the time, but I don't think this idea is particularly crazy. Usability-wise, something you can talk to (in your own "low resource" language) is a winner.
The catch is that you have users without either the sophistication to size up how trustworthy this model is or to do out-of-band verification (remember: no internet). I think a RAG model serving answers from Wikipedia could be suitably safe, but we'd have to do some user research to know more.
A future version of an ebook library would be an offline large language model? So like, instead of downloading Moby Dick, you can ask for it to be generated?
There is definitely interest in providing off-grid communities easy access to lightweight LLMs... one idea is to use something like: https://github.com/mlc-ai/web-llm to deliver it in a browser
Oh I see. Was confused. The “this” in OP’s comment is an offline library. An offline LLM would be nice too.
Yeah, to clarify, I didn't mean it would be a direct replacement for literature, just that on the same theme of of providing a resource in places where connectivity is an issue, an LLM is a reasonable way to compress a whole lot of human knowledge into a relatively small space in a usable form.