Re:Mesh LLM: distributed AI computing on iroh
by woadwarrior01 on Saturday July 11, 2026 @07:40PM
(#48876898)
Perf should be fairly straightforward to ballpark. You'll need to transfer roughly 2 . hidden_size . num_shards bytes over the network per token during autoregressive decoding. And divide that number by chunk size during prefill.
[ Parent
| on HN ]
There are no comments on this comment yet.