News for hackers. Stuff that matters.
Re:Mesh LLM: distributed AI computing on iroh by woadwarrior01 on Saturday July 11, 2026 @07:40PM (#48876898)
Perf should be fairly straightforward to ballpark. You'll need to transfer roughly 2 . hidden_size . num_shards bytes over the network per token during autoregressive decoding. And divide that number by chunk size during prefill.

There are no comments on this comment yet.