Re:Cerebras Code
by Havoc on Friday August 01, 2025 @09:20PM
(#44764183)
This seems to be rate limited by message not token so the lack of cache may matter less
[ Parent
| on HN ]
The lack of caching causes the price to increase for each message or tool call in a chat because you need to send the entire history back after every tool call. Because there isn’t any discount for cached tokens you’re looking at very expensive chat threads.
Re:Cerebras Code
by NitpickLawyer on Saturday August 02, 2025 @01:38AM
(#44765131)
Yes, but the new "thing" now is "agentic" where the driver is "tool use". So at every point where the LLM decides to make a tool use, there is a new request that gets sent. So a simple task where the model needs to edit one function down the tree, there might be 10 calls - 1st with the task, 2-5 for "read_file", then the model starts writing code, 6-7 trying to run the code, 8 fixing something, and so on...
Re:Cerebras Code
by itsafarqueue on Saturday August 02, 2025 @02:31AM
(#44765276)
Yup. If you’ve ever watched a 60+ minute agent loop spawning sub agents, your “one message” prompt leaves you several hundred messages in the hole.
No it’s by token. The FAQ says this:
> Actual number of messages per day depends on token usage per request. Estimates based on average requests of ~8k tokens each for a median user.
https://cerebras-inference.help.usepylon.com/articles/346886...