News for hackers. Stuff that matters.

A comment on M2 Ultra can run 128 streams of Llama 2 7B in parallel. Parent comment

Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by hamandcheese on Wednesday October 11, 2023 @07:32PM (#37851734)
I think you need one motivated developer who likes to tinker with hardware for a few hours a month.

Given how popular homelabs are, I don't think this would be too hard to find.

[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by lghh on Wednesday October 11, 2023 @09:56PM (#37852633)
I don’t want to build a business around a dev who likes to “tinker with hardware”.
[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by hamandcheese on Thursday October 12, 2023 @12:56AM (#37853590)
Why the scare quotes? It's a skill like any other. Startups are built by folks who wear many hats.
[ Parent | on HN ]