News for hackers. Stuff that matters.
Re:OpenAI ChatGPT: Optimizing Language Models for Dialogue by Vetch on Wednesday November 30, 2022 @06:30PM (#33809596)
I would like to say FlanT5 11B which is 45GB but my experience is if you're using huggingface the usual way, it can initially take up to 2x the memory of the model to load.

GPT-JT was released recently and seems interesting but I haven't tried it. If you're focused on scientific domain and want to do Open book Q/A, summarization, keyword extraction etc. Galactica 6B parameter version might be worth checking out.

There are no comments on this comment yet.