Ask HN: How to train and host an LLM for internal docs

Re:Ask HN: How to train and host an LLM for internal docs by nborwankar on Thursday June 29, 2023 @08:52PM (#36529067)

The LLM needs to be run locally. So it can’t be a GPT API or an API for some cloud LLM. The docs can’t leave the perimeter. It can be a Pretrained model that is being “extended” (is there such a thing?) with the new docs. Alternately it can be an existing LLM trained from scratch on these docs but I’m not sure that’s the right approach.

[ Parent | on HN ]

Re:Ask HN: How to train and host an LLM for internal docs by quickthrower2 on Friday June 30, 2023 @12:00AM (#36530368)

How local? As in on premise? Or could it run on a cloud VM?

[ Parent | on HN ]

Re:Ask HN: How to train and host an LLM for internal docs by nborwankar on Friday June 30, 2023 @05:30PM (#36542594)

Cloud VM possible, not sure. Is it possible to run on HuggingFace without compromising data privacy?On-premise preferred. Capital cost to get to “Hello world” will probably convince principals to use HuggingFace if data privacy concerns can be met.

[ Parent | on HN ]