News for hackers. Stuff that matters.

A comment on How fast can one reasonably expect to get inference on a ~70B model?. Parent comment

Re:How fast can one reasonably expect to get inference on a ... by yungtriggz on Monday May 27, 2024 @02:13PM (#40493148)
yea reckon vllm is the way to go. cheers
[ Parent | on HN ]

There are no comments on this comment yet.