Re:Ask HN: How are you using GPT to be productive?
by sillysaurusx on Saturday March 25, 2023 @08:27AM
(#35301980)
> The amount of resources required to run this on personal machines is still prohibitively high. I saw in one of your posts you mentioned you use 8xA100s. That's a crazy amount of compute unreachable by most people
FWIW LLaMA 65B can run on a single MacBook Pro now. Things move crazy fast. (Or did, before Facebook started DMCA'ing everyone.)
I did a bad job of explaining that personal GPUs will be sufficient in the near future. Thanks for pointing that out.
> thanks for leading the way, and spreading the word about why self-hosting these tools is important. I hope that our vision becomes a reality for many soon.
Thanks for talking about the issue at all. The whole reason I got into AI was to run these myself. It'll be a shame if only massive corporations can run models anyone cares about.
FWIW LLaMA 65B can run on a single MacBook Pro now. Things move crazy fast. (Or did, before Facebook started DMCA'ing everyone.)
I did a bad job of explaining that personal GPUs will be sufficient in the near future. Thanks for pointing that out.
> thanks for leading the way, and spreading the word about why self-hosting these tools is important. I hope that our vision becomes a reality for many soon.
Thanks for talking about the issue at all. The whole reason I got into AI was to run these myself. It'll be a shame if only massive corporations can run models anyone cares about.