Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

Re:Rio de Janeiro's "homegrown" LLM appears to be a merge of... by itkovian_ on Monday June 15, 2026 @12:28AM (#48536596)

This is called linear mode connectivity and seems to work for almost every large model. So well that in most cases it’s an explicit part of the training process; do many training ‘branches’ then merge then continue.

It is not understood why it works so well.

[ Parent | on HN ]

Re:Rio de Janeiro's "homegrown" LLM appears to be a merge of... by teravor on Monday June 15, 2026 @02:23AM (#48537326)

is that actually how they train them in the datacenter? the trillion sized weight vector gets cloned and sent off to groups of GPUs and averaged after?

[ Parent | on HN ]