Re:Covenant-72B: Pre-Training a 72B LLM with Trustless Peers...
by Kave0ne on Wednesday March 11, 2026 @09:15AM
(#47335162)
The Byzantine fault tolerance question here is interesting. With 72B parameters trained across untrusted peers, even a small fraction of malicious nodes could introduce subtle gradient poisoning that degrades model quality in non-obvious ways. Curious how they handle the verification overhead at scale - cryptographic proofs on gradient updates would add significant latency. Is the threat model just Sybil attacks, or also honest-but-curious nodes leaking gradient information?
[ Parent
| on HN ]
There are no comments on this comment yet.