News for hackers. Stuff that matters.
Re:PowerInfer: Fast Large Language Model Serving with a Cons... by LoganDark on Wednesday December 20, 2023 @06:54PM (#38715258)
> 24-core cpu

Pretty sure it's an 8-core CPU, or at least that's how they're using it. Their demo videos show only 8 threads in use, probably because the "efficiency cores" would negate some of the performance win.

There are no comments on this comment yet.