News for hackers. Stuff that matters.
Re:Claude was trained on protein sequences at scale by dfdydx on Wednesday November 18, 2026 @03:53AM (#50543682)
I guess predicting the next amino acid is about as useful as predicting the next word in a sentence - that is to say the task itself is not interesting but a. there is plenty of data to train on that does not need to be labeled by a human and b. hopefully this meaningfully improve performance on tasks we care about.
Re:Claude was trained on protein sequences at scale by throw310822 on Wednesday November 18, 2026 @04:12AM (#50543852)
They both depend on how much "grokking" you can get behind it. Next token prediction can go from spitting out the most common token after the previous one, to a deep understanding of the text that precedes, from syntax to semantics.
Re:Claude was trained on protein sequences at scale by zodwick on Wednesday November 18, 2026 @09:52AM (#50547021)
good point on not needing labeling, hence proly makes sense to throw it in in pretraining and see if useful capabilities emerge. Funny how a lot of this is still a black box