How 'Open' Is OpenAI, Really?

Re:How 'Open' Is OpenAI, Really? by FooBarWidget on Monday March 13, 2023 @02:34AM (#35131912)

ChatGPT can run on our laptops? How? The only thing I could find was that it can't because it requires 128 GB VRAM.

[ Parent | on HN ]

Re:How 'Open' Is OpenAI, Really? by recuter on Monday March 13, 2023 @02:37AM (#35131932)

https://github.com/ggerganov/llama.cpp/issues/39

40GB for the big llama.

[ Parent | on HN ]

Re:How 'Open' Is OpenAI, Really? by FooBarWidget on Monday March 13, 2023 @02:47AM (#35132003)

But that's not ChatGPT?

[ Parent | on HN ]

Re:How 'Open' Is OpenAI, Really? by Vespasian on Monday March 13, 2023 @03:02AM (#35132102)

Not it isn't but what the OP is saying that with the llama models (and others) being runable on consumer hardware it's only a matter of time until we get something that's "good enough" for many use cases.

Maybe not first of the class but there is a lot value to be found in the middle ground and it's good for the market as a whole.

[ Parent | on HN ]

Re:How 'Open' Is OpenAI, Really? by ShamelessC on Monday March 13, 2023 @02:56AM (#35132062)

No one said ChatGPT. It was a vague reference to the recent LLaMA developments from the community.

[ Parent | on HN ]

Re:How 'Open' Is OpenAI, Really? by karmasimida on Monday March 13, 2023 @03:10AM (#35132183)

Even biggest Llama is like 1/3 size of ChatGPT

[ Parent | on HN ]

Re:How 'Open' Is OpenAI, Really? by recuter on Monday March 13, 2023 @03:17AM (#35132234)

Might as well compare it to AlpacaGPT, they are trained different. Carefully prompted the output shows flashes of excellence and really whips ass and it is still early days.

We have almost closed the gap in no time whereas on ChatGPT day 1 it seemed they were years ahead. Hopefully there will be more models and developments.

[ Parent | on HN ]

Re:How 'Open' Is OpenAI, Really? by ly3xqhl8g9 on Monday March 13, 2023 @02:58AM (#35132073)

LLaMa [1] works on Apple Mx processors due to unified memory. The 7B model works on a 8GB Apple MacBook Pro M1, ~40 seconds/token.

[1] https://dev.l1x.be/posts/2023/03/12/using-llama-with-m1-mac

[ Parent | on HN ]

Re:How 'Open' Is OpenAI, Really? by recuter on Monday March 13, 2023 @03:07AM (#35132158)

I get 220ms/token on an M1 8GB Macbook Air, so ChatGPT speeds. on a 32GB M1 Max it babbles faster than my ex-gf.

[ Parent | on HN ]

Re:How 'Open' Is OpenAI, Really? by ly3xqhl8g9 on Monday March 13, 2023 @03:15AM (#35132227)

Weird, I ran it alongside quite a few other processes, didn't expect to even be able to run and didn't bother to optimize it in any meaningful way.

[ Parent | on HN ]

Re:How 'Open' Is OpenAI, Really? by mach1ne on Monday March 13, 2023 @04:02AM (#35132528)

Not ChatGPT, but LLaMa. Not as good as ChatGPT, but seeing the pace of progress when performance has been demonstrated, we'll have ChatGPT running on our laptops by summer. End of the year at the latest.

[ Parent | on HN ]