News for hackers. Stuff that matters.
Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by gavinray on Sunday July 23, 2023 @02:58AM (#36833140)
Do it yourself in ~20 lines of code with LlamaIndex.

You give it a directory containing documents and ask it to build an index and vector data embeddings over the documents

Then you can use this index with models like ChatGPT

Tutorial here shows the end to end process

https://gpt-index.readthedocs.io/en/latest/getting_started/s...

Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by whynotmaybe on Sunday July 23, 2023 @08:37AM (#36834764)
Does it work well with content in other languages than English?

Especially for French and Spanish.

Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by DonsDiscountGas on Monday July 24, 2023 @06:46PM (#36855309)
I dunno about this tool specifically, but whenever I've tried LLMs in spanish they work just as well as in english. You can also change LLMs to something explicitly multi-lingual, like BLOOM

https://huggingface.co/blog/bloom

Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by fernirello on Sunday July 23, 2023 @04:35PM (#36839555)
From a confidentiality viewpoint, isn't this equivalent to uploading my docs to ChatPDF? (Genuine question, I'm not well versed in LLMs.) Because I'm uploading the embeddings to OpenAI or a similar service provider-if their chatbot can then answer my questions about my docs, wouldn't the informational content also allow it to answer any question asked by anybody other than me? E.g., "what's the document about?", "what's the PII of principal X?", etc.