Re:Claude was trained on protein sequences at scale
by tstactplsignore on Wednesday November 18, 2026 @07:48AM
(#50545677)
It is a negative control in the experiment. Shuffling is a better control than random because it accounts for the raw rates of amino acids in each protein. For example W is very rare, so Claude could beat a truly random generator just by guessing not W. But the signal seen on real sequences is much stronger than that, so it tells you that Claude is doing something more than just guessing 'not W' on real sequences.
[ Parent
| on HN ]
Re:Claude was trained on protein sequences at scale
by octoberfranklin on Wednesday November 18, 2026 @10:11AM
(#50547265)
But the signal seen on real sequences is much stronger than that
But the shuffled sequences aren't real sequences!
Sorry if I'm being daft here, but I still don't understand what the source of truth is for the shuffled sequence tests.
Re:Claude was trained on protein sequences at scale
by wavemode on Wednesday November 18, 2026 @10:43AM
(#50547697)
[delayed]
Re:Claude was trained on protein sequences at scale
by octoberfranklin on Wednesday November 18, 2026 @01:13PM
(#50549870)
Oh! So a lower score on the "shuffled" sequence is more validating of the experimental hypothesis.
I wish the article had stated this clearly. It seems to imply the opposite, which is of course false.