News for hackers. Stuff that matters.

A comment on M2 Ultra can run 128 streams of Llama 2 7B in parallel. Parent comment

Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by chmod775 on Thursday October 12, 2023 @03:35AM (#37854462)
Nowadays GPUs factory defaults try to squeeze out the last bit of performance for a huge cost in power.

You can run them at half the power usage and only lose a fraction of the performance - at least in gaming. Try for AI tasks.

[ Parent | on HN ]

There are no comments on this comment yet.