News for hackers. Stuff that matters.

A comment on LoRA: Low-Rank Adaptation of Large Language Models. Parent comment

Re:LoRA: Low-Rank Adaptation of Large Language Models by numlocked on Friday March 24, 2023 @12:16PM (#35291251)
Interesting. I wonder what the training cost was for:

https://huggingface.co/EleutherAI/gpt-neox-20b

Perhaps it’s in the paper…

[ Parent | on HN ]
Re:LoRA: Low-Rank Adaptation of Large Language Models by michaelhartm on Friday March 24, 2023 @03:11PM (#35294040)
They used the 6b GPT4-J, not 20B. That's what's interesting, it's a smallish large language model :).
[ Parent | on HN ]
Re:LoRA: Low-Rank Adaptation of Large Language Models by dragonwriter on Friday March 24, 2023 @04:45PM (#35295199)
GPT-J, not GPT4-J.
[ Parent | on HN ]