Computing Inside an AI

Re:Computing Inside an AI by ilaksh on Sunday December 15, 2024 @12:29AM (#42421687)

I think that Cerebras and Groq would be fun to experiment with using normal LLMs for generating interfaces on the fly, since they are so fast.

[ Parent | on HN ]

Re:Computing Inside an AI by FezzikTheGiant on Sunday December 15, 2024 @12:49AM (#42421757)

what's the cost difference between groq/cerebras vs using something else for inferencing open source models? I'm guessing the speed comes at a cost?

[ Parent | on HN ]

Re:Computing Inside an AI by ilaksh on Sunday December 15, 2024 @12:53AM (#42421776)

I don't know off the top of my head, only played with it a little not seriously.

[ Parent | on HN ]

Re:Computing Inside an AI by FezzikTheGiant on Sunday December 15, 2024 @01:41AM (#42421954)

fair enough

[ Parent | on HN ]

Re:Computing Inside an AI by el_isma on Sunday December 15, 2024 @08:35AM (#42423311)

0.6/1$ per M tokens in groq/cerebras vs 0.3$ per M tokens in deepinfra (for llama 3.3 70b)

But note the free tiers for groq and cerebras are very generous.

[ Parent | on HN ]