News for hackers. Stuff that matters.
Re:Cerebras Code by Havoc on Friday August 01, 2025 @09:20PM (#44764183)
This seems to be rate limited by message not token so the lack of cache may matter less
Re:Cerebras Code by Flux159 on Friday August 01, 2025 @09:51PM (#44764294)
The lack of caching causes the price to increase for each message or tool call in a chat because you need to send the entire history back after every tool call. Because there isn’t any discount for cached tokens you’re looking at very expensive chat threads.
Re:Cerebras Code by NitpickLawyer on Saturday August 02, 2025 @01:38AM (#44765131)
Yes, but the new "thing" now is "agentic" where the driver is "tool use". So at every point where the LLM decides to make a tool use, there is a new request that gets sent. So a simple task where the model needs to edit one function down the tree, there might be 10 calls - 1st with the task, 2-5 for "read_file", then the model starts writing code, 6-7 trying to run the code, 8 fixing something, and so on...
Re:Cerebras Code by itsafarqueue on Saturday August 02, 2025 @02:31AM (#44765276)
Yup. If you’ve ever watched a 60+ minute agent loop spawning sub agents, your “one message” prompt leaves you several hundred messages in the hole.
Re:Cerebras Code by andhuman on Saturday August 02, 2025 @02:48AM (#44765345)
No it’s by token. The FAQ says this:

> Actual number of messages per day depends on token usage per request. Estimates based on average requests of ~8k tokens each for a median user.

https://cerebras-inference.help.usepylon.com/articles/346886...

Re:Cerebras Code by jtbayly on Saturday August 02, 2025 @01:01PM (#44769241)
How did you find that? Are you sure it applies to Cerebras Code Pro or Max?