News for hackers. Stuff that matters.

A comment on PowerInfer: Fast Large Language Model Serving with a Consumer-Grade GPU [pdf].

Re:PowerInfer: Fast Large Language Model Serving with a Cons... by Animats on Tuesday December 19, 2023 @06:19PM (#38703333)
That's an impressive result. LLMs should soon be much cheaper to run.
[ Parent | on HN ]

There are no comments on this comment yet.