Re:PowerInfer: Fast Large Language Model Serving with a Cons...
by LoganDark on Wednesday December 20, 2023 @06:54PM
(#38715258)
> 24-core cpu
Pretty sure it's an 8-core CPU, or at least that's how they're using it. Their demo videos show only 8 threads in use, probably because the "efficiency cores" would negate some of the performance win.
Pretty sure it's an 8-core CPU, or at least that's how they're using it. Their demo videos show only 8 threads in use, probably because the "efficiency cores" would negate some of the performance win.