News for hackers. Stuff that matters.

A comment on Replit - How to train your own LLM Models. Parent comment

Re:Replit - How to train your own LLM Models by dkhudia on Thursday April 20, 2023 @01:07AM (#35636621)
Disclaimer: I work for MosaicML (MosaicML is the creator of the training platform used by Replit).

Training these models from scratch on your domain specific data is not as expensive as one might think. We have provided some cost estimates in our blogs.

https://www.mosaicml.com/blog/mosaicbert

https://www.mosaicml.com/blog/training-stable-diffusion-from...

https://www.mosaicml.com/blog/gpt-3-quality-for-500k

[ Parent | on HN ]
Re:Replit - How to train your own LLM Models by ftxbro on Thursday April 20, 2023 @03:18AM (#35637256)
Thank you this is very interesting!
[ Parent | on HN ]
Re:Replit - How to train your own LLM Models by moltar on Thursday April 20, 2023 @03:51AM (#35637437)
Do you have any examples on how to train a model that can write code but in a specific domain? Eg I only want to train it on a specific set of code. Eg let’s say functional React components in TypeScript.
[ Parent | on HN ]
Re:Replit - How to train your own LLM Models by dskhudia on Thursday April 20, 2023 @12:23PM (#35642390)
We recently released 1B parameter model trained on a mix of data.[1] If you got your domain-specific data, our platform can cover the rest.

[1]: https://twitter.com/jefrankle/status/1649060478910357504?s=4...

[ Parent | on HN ]
Re:Replit - How to train your own LLM Models by moltar on Thursday April 20, 2023 @12:31PM (#35642496)
But do you have any examples of how to do this? I am a pretty seasoned dev, but never trained a model before :)
[ Parent | on HN ]