News for hackers. Stuff that matters.

A comment on Mistral: Our first AI endpoints are available in early access. Parent comment

Re:Mistral: Our first AI endpoints are available in early ac... by modeless on Monday December 11, 2023 @11:09AM (#38601813)
Are you using batch size 1 with LLMs? Larger batch sizes get much higher utilization.
[ Parent | on HN ]

There are no comments on this comment yet.