News for hackers. Stuff that matters.
Re:Claude was trained on protein sequences at scale by zodwick on Wednesday November 18, 2026 @02:34AM (#50542989)
Is this a valuable skill ? Sorry, bit of a noob question ig but have been seeing a lot of buzz around protein sequences in general
Re:Claude was trained on protein sequences at scale by gilleain on Wednesday November 18, 2026 @02:50AM (#50543096)
Do you mean 'skill for a human' or 'Skill for an LLM'? :)

I used to do bioinformatics, although more of the structure side than sequence. Like an information or understanding, it depends on what outcome you want that might benefit from the skill.

I suppose that predicting the next amino acid in a sequence would be useful for, say, reconstructing a protein sequence from fragments. Or constructing evolutionary trees of sequences by predicting the common ancestors?

Re:Claude was trained on protein sequences at scale by zodwick on Wednesday November 18, 2026 @03:24AM (#50543405)
I'm not sure how to phrase this correctly sorry, But I suppose the question I have is what comes after say "reconstructing a protein sequence from fragments" -> like what how does one go from this to developing a drug or enzyme. Is being able to predict the next amino acid useful in this process ?

Linking to ML space, is this analogous to knowing the model's weights byheart ? Traditionally this has been not useful to humans and machines, but recently some LLMs have started showing capability to directly edit the model weights to alter the behavious of smaller models. Is that a completely wrong analogy ?

Also I guess what I am partly curious is on how this data ended up in the model. Was it part of the pretraining corpus or was there an RL environment which specifically rewarded this behaviour

Re:Claude was trained on protein sequences at scale by gilleain on Wednesday November 18, 2026 @03:39AM (#50543551)
Ok the next steps beyond predicting a sequence would then be modelling the structure - which requires something like AlfaFold or 'traditional' (as it were) software for molecular mechanics and dynamics.

How useful the predicted sequence is for the predicted structure - and predicted function - no idea! There are already several layers of prediction here.

Sorry, I definitely do not know enough about LLMs to understand the analogy to weights. You can predict the effects of mutations on sequences, I think? As in, if I change this Q to an E will it make the enzyme faster (??). Perhaps you could LLM-generate a plausible member of a family of enzymes with a particular function, given fragments of an existing protein (???).

No idea how the data got in the model. It seems like the study that Xinming and Chen have done was designed to eliminate sources from the general corpus of knowledge - like any old science papers - b choosing obscure and rare sequences that might only be found in a sequence db.

Re:Claude was trained on protein sequences at scale by zodwick on Wednesday November 18, 2026 @09:58AM (#50547085)
Unfortunately I don't have any expertise in bioinformatics and hence can't provide good feedback. Almost as if we need something with good understanding of both fields :)

jokes apart I do hope the LLMs ability to being a master at multiple unrelated fields unlock valuable insights

On the data getting in the model - I suspect: as the models have gotten bigger, the labs have started including more info from sequence databases of some sort

Re:Claude was trained on protein sequences at scale by dfdydx on Wednesday November 18, 2026 @03:53AM (#50543682)
I guess predicting the next amino acid is about as useful as predicting the next word in a sentence - that is to say the task itself is not interesting but a. there is plenty of data to train on that does not need to be labeled by a human and b. hopefully this meaningfully improve performance on tasks we care about.
Re:Claude was trained on protein sequences at scale by throw310822 on Wednesday November 18, 2026 @04:12AM (#50543852)
They both depend on how much "grokking" you can get behind it. Next token prediction can go from spitting out the most common token after the previous one, to a deep understanding of the text that precedes, from syntax to semantics.
Re:Claude was trained on protein sequences at scale by zodwick on Wednesday November 18, 2026 @09:52AM (#50547021)
good point on not needing labeling, hence proly makes sense to throw it in in pretraining and see if useful capabilities emerge. Funny how a lot of this is still a black box
Re:Claude was trained on protein sequences at scale by dlougheed on Wednesday November 18, 2026 @06:28AM (#50544975)
an LLM-predicted evolutionary tree seems monumentally useless, since it'll be unstable and you have no guarantees/metrics on any property or quality of the tree itself (parsimony, likelihood, estimated TMRCA, etc.)
Re:Claude was trained on protein sequences at scale by gilleain on Wednesday November 18, 2026 @06:32AM (#50545014)
Totally fair, I'm well beyond my comfort zone in bioinformatics :) I should stick to structure.
Re:Claude was trained on protein sequences at scale by zodwick on Wednesday November 18, 2026 @10:01AM (#50547121)
out of my comfort zone, but out of curiousity; are these parameters ( parsimony, likelihood, estimated TMRCA, etc.) verifiable easily ? If so I wouldn't dismiss capabilities of LLMs in predicting these. All tasks that can we RL'd will be RL'd imho.