News for hackers. Stuff that matters.
Re:Claude was trained on protein sequences at scale by t0mek on Wednesday November 18, 2026 @09:15AM (#50546583)
In "Empire of AI", the author mentions that when OpenAI included public GitHub repositories in its training data, the resulting model achieved better results even in fields unrelated to programming.

Aside from the obvious applications in biochemistry, could protein sequences also be used as a new training data source to improve overall model quality? If that's true, it's quite fascinating that we can take highly complex data we don't fully understand and use it to improve LLM training, simply because it contains so many significant structures and relationships.

Re:Claude was trained on protein sequences at scale by schiffern on Wednesday November 18, 2026 @09:33AM (#50546781)
TL;DR visit weird museums and stare into fires?
Re:Claude was trained on protein sequences at scale by phkahler on Wednesday November 18, 2026 @09:36AM (#50546815)
Software is used in all fields, and github issues and PRs are going to contain detailed reasoning about all sorts of things when adding features or fixing bugs.
Re:Claude was trained on protein sequences at scale by fritzo on Wednesday November 18, 2026 @10:59AM (#50547913)
Right

> GitHub repositories ... in fields unrelated to programming

About as surprising as training on books and improving in fields other than writing.

Re:Claude was trained on protein sequences at scale by cedws on Wednesday November 18, 2026 @11:28PM (#50556089)
Also many people use GitHub as their personal hard drive and push all sorts of random stuff, including PDFs that could come from Arxiv, etc.
Re:Claude was trained on protein sequences at scale by acka on Thursday November 19, 2026 @07:00PM (#50568350)
It is important to note that there was a time when private repositories were not available to users on the free GitHub plan. For those wanting to use GitHub as their personal hard drive, this meant that the repositories containing their documents needed to be public.
Re:Claude was trained on protein sequences at scale by origini on Wednesday November 18, 2026 @11:45AM (#50548534)
We have seen it happening in math and physics.

We have seen that LLMs are quite good in reading binary and changing binary directly.

Could be possible.

Re:Claude was trained on protein sequences at scale by dekhn on Wednesday November 18, 2026 @02:35PM (#50550986)
It seems not entirely unreasonable to expect that training on large biological sequence repositories could uncover structure that is applicable to information theory. There has been a great deal of cross-pollination between those two fields.