Ask HN: How to train and host an LLM for internal docs

Re:Ask HN: How to train and host an LLM for internal docs by samspenc on Thursday June 29, 2023 @07:15PM (#36528131)

Are you building your own "LLM" for this use case (more complex and expensive)? Or just generating embeddings for OpenAI or other LLMs that you integrate your own data with (easier and less expensive) https://openai.com/blog/introducing-text-and-code-embeddings

[ Parent | on HN ]

Re:Ask HN: How to train and host an LLM for internal docs by nborwankar on Thursday June 29, 2023 @08:52PM (#36529067)

The LLM needs to be run locally. So it can’t be a GPT API or an API for some cloud LLM. The docs can’t leave the perimeter. It can be a Pretrained model that is being “extended” (is there such a thing?) with the new docs. Alternately it can be an existing LLM trained from scratch on these docs but I’m not sure that’s the right approach.

[ Parent | on HN ]

Re:Ask HN: How to train and host an LLM for internal docs by quickthrower2 on Friday June 30, 2023 @12:00AM (#36530368)

How local? As in on premise? Or could it run on a cloud VM?

[ Parent | on HN ]

Re:Ask HN: How to train and host an LLM for internal docs by nborwankar on Friday June 30, 2023 @05:30PM (#36542594)

Cloud VM possible, not sure. Is it possible to run on HuggingFace without compromising data privacy?On-premise preferred. Capital cost to get to “Hello world” will probably convince principals to use HuggingFace if data privacy concerns can be met.

[ Parent | on HN ]