News for hackers. Stuff that matters.

A comment on How fast can one reasonably expect to get inference on a ~70B model?.

Re:How fast can one reasonably expect to get inference on a ... by uptownfunk on Friday May 24, 2024 @08:29PM (#40471695)
Dumb q- have you profiled the inference execution? Where are the bottlenecks you are observing?
[ Parent | on HN ]

There are no comments on this comment yet.