News for hackers. Stuff that matters.

A comment on M2 Ultra can run 128 streams of Llama 2 7B in parallel. Parent comment

Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by sitkack on Wednesday October 11, 2023 @07:36PM (#37851766)
There are multiple riscv based solutions that will be out in 12-18 months. But for now, getting Apple hardware is the best solution.
[ Parent | on HN ]

There are no comments on this comment yet.