News for hackers. Stuff that matters.

A comment on How 'Open' Is OpenAI, Really?. Parent comment

Re:How 'Open' Is OpenAI, Really? by ly3xqhl8g9 on Monday March 13, 2023 @02:58AM (#35132073)
LLaMa [1] works on Apple Mx processors due to unified memory. The 7B model works on a 8GB Apple MacBook Pro M1, ~40 seconds/token.

[1] https://dev.l1x.be/posts/2023/03/12/using-llama-with-m1-mac

[ Parent | on HN ]
Re:How 'Open' Is OpenAI, Really? by recuter on Monday March 13, 2023 @03:07AM (#35132158)
I get 220ms/token on an M1 8GB Macbook Air, so ChatGPT speeds. on a 32GB M1 Max it babbles faster than my ex-gf.
[ Parent | on HN ]
Re:How 'Open' Is OpenAI, Really? by ly3xqhl8g9 on Monday March 13, 2023 @03:15AM (#35132227)
Weird, I ran it alongside quite a few other processes, didn't expect to even be able to run and didn't bother to optimize it in any meaningful way.
[ Parent | on HN ]