News for hackers. Stuff that matters.

A comment on Open source ChatGPT with LLaMA implementation trains 15x faster.

Re:Open source ChatGPT with LLaMA implementation trains 15x ... by rnosov on Monday February 27, 2023 @09:41AM (#34956945)
Looks like they are making ChatGPT clone that would be possible to run a single GPU. HN dream come true!
[ Parent | on HN ]
Re:Open source ChatGPT with LLaMA implementation trains 15x ... by wcarss on Monday February 27, 2023 @09:44AM (#34956979)
ChatGPT's stable diffusion moment?
[ Parent | on HN ]
Re:Open source ChatGPT with LLaMA implementation trains 15x ... by Taek on Monday February 27, 2023 @11:24AM (#34958278)
Can't have a stable diffusion moment if you refuse to release the weights to the general public. Stable diffusion only got to where it is because 10,000 people with otherwise zero reputation were able to play around with the code and models.

LLaMA is still only available to the elite.

[ Parent | on HN ]
Re:Open source ChatGPT with LLaMA implementation trains 15x ... by sinity on Friday March 03, 2023 @01:35AM (#35006522)
It was released on 4chan recently :)

files_catbox_moe[slash]o8a7xw(dot)torrent

[ Parent | on HN ]
Re:Open source ChatGPT with LLaMA implementation trains 15x ... by swyx on Monday February 27, 2023 @11:49AM (#34958686)
lol good luck running a 13B model on a single GPU
[ Parent | on HN ]
Re:Open source ChatGPT with LLaMA implementation trains 15x ... by ddren on Monday February 27, 2023 @12:38PM (#34959403)
Seeing the performance of implementations like FlexGen [1], I don't think it would be entirely unreasonable to run a 13B model on a single GPU for personal usage purposes. You are not going to a run a public service out of it, but it probably would be good enough to run your own ChatGPT or Copilot locally.

[1]: https://github.com/FMInference/FlexGen

[ Parent | on HN ]
Re:Open source ChatGPT with LLaMA implementation trains 15x ... by visarga on Monday February 27, 2023 @11:58AM (#34958819)
You need a RTX 3090 24gb
[ Parent | on HN ]
Re:Open source ChatGPT with LLaMA implementation trains 15x ... by flangola7 on Monday February 27, 2023 @10:56PM (#34965432)
It would not be a HN dream for long, given the implications it has for the internet
[ Parent | on HN ]