Ask HN: How do I train a custom LLM/ChatGPT on my own documents?

Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by Nevin1901 on Sunday July 23, 2023 @09:15AM (#36835007)

I'm building a service called https://useftn.com which allows people to do exactly that. You just upload your dataset and we'll fine tune Llama 2-7B for you on that data. The model works with huggingface and all of the mainstream nlp frameworks.

It's in its really early stages right now (mainly just looking to learn/help people), so if you have your data in a specific format I'll be happy to code something up to make it work with your data.

(Also Disclaimer: I own this service)

[ Parent | on HN ]

Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by potamic on Sunday July 23, 2023 @02:16PM (#36838072)

Any estimate for how long it would take to fine tune against a given volume of data?

[ Parent | on HN ]

Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by Nevin1901 on Sunday July 23, 2023 @07:18PM (#36841008)

Currently not but I'm actively working on that feature. For now if the costs are getting too high, you can just stop training and we'll export the last saved checkpoint.

[ Parent | on HN ]

Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by kirdiekirdie on Friday July 28, 2023 @03:47AM (#36904073)

How do you finetune a dataset on LLaMA2-7b with an A10 with 24 GB? I thought you need to load the dataset 4 times for training, so with 16 bit weights this would be 7 * 10^9 * 2 bytes * 4 = 56 GB GPU RAM. Have you found a way to train with 4 bit weights? And would a single textbook be enough text for a meaningful finetuning? I am worried that a model that is trained on billions of documents will not adapt strongly enough to a comparatively small document.

[ Parent | on HN ]