News for hackers. Stuff that matters.

A comment on Computing Inside an AI. Parent comment

Re:Computing Inside an AI by FezzikTheGiant on Sunday December 15, 2024 @12:49AM (#42421757)
what's the cost difference between groq/cerebras vs using something else for inferencing open source models? I'm guessing the speed comes at a cost?
[ Parent | on HN ]
Re:Computing Inside an AI by ilaksh on Sunday December 15, 2024 @12:53AM (#42421776)
I don't know off the top of my head, only played with it a little not seriously.
[ Parent | on HN ]
Re:Computing Inside an AI by FezzikTheGiant on Sunday December 15, 2024 @01:41AM (#42421954)
fair enough
[ Parent | on HN ]
Re:Computing Inside an AI by el_isma on Sunday December 15, 2024 @08:35AM (#42423311)
0.6/1$ per M tokens in groq/cerebras vs 0.3$ per M tokens in deepinfra (for llama 3.3 70b)

But note the free tiers for groq and cerebras are very generous.

[ Parent | on HN ]