Ask HN: What laptop would you recommend for running LLM models locally?

Posted by josefslerka on Saturday May 06, 2023 @03:21AM
from the ask-hn dept.

( 3 comments | on HN )

Re:Ask HN: What laptop would you recommend for running LLM m... by ftxbro on Saturday May 06, 2023 @03:54AM (#35839176)

I'm not an apple fanboy but I would consider some of the apple laptops, because they have some kind of sharing of RAM between their AI accelerators and their main memory. This lets them load bigger models than normal, which can be the limiting factor in running local LLM models.

[ Parent | on HN ]

Re:Ask HN: What laptop would you recommend for running LLM m... by josefslerka on Saturday May 06, 2023 @04:31AM (#35839375)

Right now I'm trying Vicuna 13b on a MacbookAir M1, 16GB and it's extremely slow. 7b is ok.

[ Parent | on HN ]

Re:Ask HN: What laptop would you recommend for running LLM m... by throwaway50606 on Saturday May 06, 2023 @05:51AM (#35839788)

I have the Air too. It's thermal throttled. Try someone's M1 Pro, or the Mac Mini. Also make sure you're using the GPU (I'm not sure but AFAIK these models can run on CPU too?).

People at my employer use maxxed out Macbook Pro M2 for their ML research needs and say you can't get better performance for that price (in a laptop).

[ Parent | on HN ]