Re:Claude was trained on protein sequences at scale
by _ink_ on Wednesday November 18, 2026 @02:53AM
(#50543122)
I expect this to be another revenue stream. My guess is that there will be an accreditation step and afterwards you get access to an unlocked model. Just as in security. And possibly it will come with a price tag.
Re:Claude was trained on protein sequences at scale
by r_lee on Wednesday November 18, 2026 @02:58AM
(#50543178)
or you have to have your organization approved so then everything will be done through approved "partners" I.e. consulting companies who add their own markup ontop of everything. we're already seeing this with Mythos
Re:Claude was trained on protein sequences at scale
by jwr on Wednesday November 18, 2026 @03:48AM
(#50543637)
This already exists with Mythos. You can apply to get approved for cybersecurity testing, which is relatively easy if your need to just test your own software. But Mythos is accessible only through the API, not through your subscription, so you will be paying quite a bit for it.
Re:Claude was trained on protein sequences at scale
by bortimus on Wednesday November 18, 2026 @09:09AM
(#50546525)
Have access to Mythos, use it as an alternative to Fable when I'm out of Fable credits. Seems to not count against it, or at least have its own limits.
Re:Claude was trained on protein sequences at scale
by chriskanan on Wednesday November 18, 2026 @07:54AM
(#50545747)
I'm told by colleagues in the life sciences and pharma space that it isn't just "safety" but more like artificial market segmentation. If one has to pay extra for guardrail removal, then that's market segmentation. Anthropic makes most money from company whales. Want the guardrails gone? Pay a lot extra over baseline. But Mythos probably had PEFT adapters, so that at least makes it a bit different. But still there is not nearly enough transparency about these things.
Re:Claude was trained on protein sequences at scale
by MisterMunchkin on Wednesday November 18, 2026 @11:39AM
(#50548466)
And oopsie whoopsie, looks like we accidentally guardrailed out all of the startups that might challenge your pharmaceutical company. Thanks for being such a loyal customer :)
Re:Claude was trained on protein sequences at scale
by asdff on Thursday November 19, 2026 @12:19AM
(#50556354)
How pretty the founders of tools like docker must be sitting right now. These ai tools are going to parrot to just use docker for the next thousand years. Any new entrants are going to die like a seedling trying to take root under the canopy of a giant tree.
Re:Claude was trained on protein sequences at scale
by hnbad on Wednesday November 18, 2026 @04:50PM
(#50552666)
They're not "triggering Opus" as in "making Opus react", they're triggering a rule in Claude that makes it drop the model from Fable to Opus, hence, triggering (a downgrade to) Opus.
It's easy to trigger these rules - just ask anything that could even remotely indicate you're working on a biological weapon (or weapons in general). It's also very prone to happening when you do anything cybersecurity-related. I'm honestly surprised I managed to have Fable help me dig through Wireshark packet captures but it delegated most of the work to Opus sub-agents anyway so maybe that's what did the trick.
Re:Claude was trained on protein sequences at scale
by dekhn on Wednesday November 18, 2026 @12:06PM
(#50548837)
I have a BAM file containing my personal genome sequence. In principle I might want to use Claude to analyze it (I do not- I don't think it would be very useful ).
Re:Claude was trained on protein sequences at scale
by mrguyorama on Wednesday November 18, 2026 @08:27PM
(#50554911)
Who did you go through to sequence your genome and how did you maintain privacy?
I'd love to do this. I'd love to know exactly which ADHD correlated mutations I have, but who the hell can I trust? Who can I trust to even run such a sequence instead of just half assing it or similar?
Re:Claude was trained on protein sequences at scale
by dekhn on Wednesday November 18, 2026 @08:44PM
(#50555045)
I used Illumina's Understanding Your Genome. It was basically a marketing tool they had to convince execs across medical industry to use Illumina sequencers in a medical context. You'd fly to San Diego, stay in a nice spa resort, get an iPad with the BAM file with your genome, a viewer app, and chat with genetic counsellors. I know enough about genomics and health that I was really interesting to see if they could tell me anything about my health that I could act on. Instead, they said I did not have a single mutation (variant) that was associated with a known disease/genome relationship (at the time, 2014, there was far less information, but BRCA and APOE1 and a number of other variants associated with diseases were known). The more I looked into how the counselors work, the less confident I was in their process ("when we see a variant that isn't in the database, we google it and skim recent papers").
As for privacy: I didn't. I uploaded my genome to PGP (Personal Genome Project)- I just noticed their website doesn't seem to work anymore, and a few minutes of Googling suggest it is no longer easy to obtain (I kept a copy). I also posted a link to HN (12 years ago) which makes it pretty easy to identify me. I weighed the facts about genomes, identity, and health, and concluded that attempting to hide my genomics sequence was not an effective strategy.
As for you doing it, I honestly don't know. If your goal is to understand your disease (or diagnose yourself), genomic data is unlikely to provide you with anything accurate and/or actionable. Sorry I can't be more optimistic- with some notable exceptions, genomic data is really not that useful for health insights.
Re:Claude was trained on protein sequences at scale
by rootsudo on Thursday November 19, 2026 @01:49AM
(#50556787)
If you have sequenced your DNA you have a vcf and more files, you can see interesting pairs, genetic defects, etc you can also go more if you dare and do some diy crispr, it isn't 100% there yet but there are some groups online that are doing some interesting testing wtih it :)
Re:Claude was trained on protein sequences at scale
by troupo on Wednesday November 18, 2026 @02:10AM
(#50542864)
You, however, will be banned for "buosecurity" research. Or, as Amodei puts ut, "you can't have opensource models because with them you can't ban users"
Re:Claude was trained on protein sequences at scale
by kypro on Wednesday November 18, 2026 @08:11AM
(#50545893)
What's the steel man argument for giving all of humanity access to advanced bio research and capabilities?
I think I'm naive sometimes when it comes to trade-offs between freedom and security, but even I'm confused at why it would be a good thing for the whole of humanity having access to this stuff...
My only argument in favour is that it seems rather certain that at some point someone will leak the weights of a model with these capabilities so it might be safer if we brought forward the risks while they are still management so people take them extremely seriously.
If we go 5-10 years and then the weights of an ultra-capable model is leaked that may be game over.
Re:Claude was trained on protein sequences at scale
by wry_xy on Wednesday November 18, 2026 @11:51AM
(#50548623)
The knowledge and capabilities are latent regardless of which humans if any have access to them. Same for their risks.
We can "protect" ourselves by limiting who has access, but:
- how often do those "trustworthy" elites use their abilities for abuse?
- it might leak
- someone else could independently obtain the capabilities
- an "act of god" might hit us without human intent
In any event, we're less prepared, because we assumed it couldn't happen. And it's not just bio.
And that's ignoring deeper questions of enforcement or censorship or surveillance.
I won't fight for a world where people are intentionally kept dumb, where the powerful are intentionally kept powerful, and where we intentionally keep ourselves vulnerable.
Re:Claude was trained on protein sequences at scale
by mrguyorama on Wednesday November 18, 2026 @08:33PM
(#50554963)
Bad guy asks OpenClaude7 for a specific sequence to kill all the black people.
OpenClaude7 provides a sequence of DNA that in theory should be able to target and kill only black people (ignore the feasibility of this, it doesn't matter)
Bad guy now has a bunch of text and zero deployable pathogens to kill all the black people.
What do you think goes next in the list of steps? Where does the dying happen? How does Bad Guy turn theory into practice?
Re:Claude was trained on protein sequences at scale
by octoberfranklin on Wednesday November 18, 2026 @02:21AM
(#50542910)
Curious: how can there be a correct prediction for the next element of a "shuffled" protein? The shuffled proteins don't occur in real life, so what makes one answer correct and another incorrect?
Re:Claude was trained on protein sequences at scale
by gilleain on Wednesday November 18, 2026 @02:56AM
(#50543148)
Perhaps. They explain why:
> Shuffling keeps the letters and destroys their order, so the difference in accuracy measures what the model knows about the sequence itself.
So by analogy, if you know a word in English starts with 'Q', you could reasonably predict the next letter would be 'u'. By shuffling, you might encounter the letter Q later on, and not successfully predict a 'u'.
Put another way, the shuffled sequences can only use frequency analysis for prediction, while the unshuffled ones have local and global order signals.
Re:Claude was trained on protein sequences at scale
by gilleain on Wednesday November 18, 2026 @03:25AM
(#50543422)
We are definitely going beyond the borders of understanding of sequences here, but I would guess the source of truth is just the expected frequency distribution of amino acids? So 'G' (glycine) is most frequent, and something like 'W' (tryptophan) is least frequent (dependent on organism?).
However I thought the point was that the _difference_ between the shuffled prediction and the unshuffled is the measure here. So shuffled is the baseline, and the gap above that is how good the prediction is.
Re:Claude was trained on protein sequences at scale
by tstactplsignore on Wednesday November 18, 2026 @07:48AM
(#50545677)
It is a negative control in the experiment. Shuffling is a better control than random because it accounts for the raw rates of amino acids in each protein. For example W is very rare, so Claude could beat a truly random generator just by guessing not W. But the signal seen on real sequences is much stronger than that, so it tells you that Claude is doing something more than just guessing 'not W' on real sequences.
Re:Claude was trained on protein sequences at scale
by gilleain on Wednesday November 18, 2026 @02:50AM
(#50543096)
Do you mean 'skill for a human' or 'Skill for an LLM'? :)
I used to do bioinformatics, although more of the structure side than sequence. Like an information or understanding, it depends on what outcome you want that might benefit from the skill.
I suppose that predicting the next amino acid in a sequence would be useful for, say, reconstructing a protein sequence from fragments. Or constructing evolutionary trees of sequences by predicting the common ancestors?
Re:Claude was trained on protein sequences at scale
by zodwick on Wednesday November 18, 2026 @03:24AM
(#50543405)
I'm not sure how to phrase this correctly sorry, But I suppose the question I have is what comes after say "reconstructing a protein sequence from fragments" -> like what how does one go from this to developing a drug or enzyme. Is being able to predict the next amino acid useful in this process ?
Linking to ML space, is this analogous to knowing the model's weights byheart ? Traditionally this has been not useful to humans and machines, but recently some LLMs have started showing capability to directly edit the model weights to alter the behavious of smaller models. Is that a completely wrong analogy ?
Also I guess what I am partly curious is on how this data ended up in the model. Was it part of the pretraining corpus or was there an RL environment which specifically rewarded this behaviour
Re:Claude was trained on protein sequences at scale
by gilleain on Wednesday November 18, 2026 @03:39AM
(#50543551)
Ok the next steps beyond predicting a sequence would then be modelling the structure - which requires something like AlfaFold or 'traditional' (as it were) software for molecular mechanics and dynamics.
How useful the predicted sequence is for the predicted structure - and predicted function - no idea! There are already several layers of prediction here.
Sorry, I definitely do not know enough about LLMs to understand the analogy to weights. You can predict the effects of mutations on sequences, I think? As in, if I change this Q to an E will it make the enzyme faster (??). Perhaps you could LLM-generate a plausible member of a family of enzymes with a particular function, given fragments of an existing protein (???).
No idea how the data got in the model. It seems like the study that Xinming and Chen have done was designed to eliminate sources from the general corpus of knowledge - like any old science papers - b choosing obscure and rare sequences that might only be found in a sequence db.
Re:Claude was trained on protein sequences at scale
by zodwick on Wednesday November 18, 2026 @09:58AM
(#50547085)
Unfortunately I don't have any expertise in bioinformatics and hence can't provide good feedback. Almost as if we need something with good understanding of both fields :)
jokes apart I do hope the LLMs ability to being a master at multiple unrelated fields unlock valuable insights
On the data getting in the model - I suspect: as the models have gotten bigger, the labs have started including more info from sequence databases of some sort
Re:Claude was trained on protein sequences at scale
by dfdydx on Wednesday November 18, 2026 @03:53AM
(#50543682)
I guess predicting the next amino acid is about as useful as predicting the next word in a sentence - that is to say the task itself is not interesting but a. there is plenty of data to train on that does not need to be labeled by a human and b. hopefully this meaningfully improve performance on tasks we care about.
Re:Claude was trained on protein sequences at scale
by throw310822 on Wednesday November 18, 2026 @04:12AM
(#50543852)
They both depend on how much "grokking" you can get behind it. Next token prediction can go from spitting out the most common token after the previous one, to a deep understanding of the text that precedes, from syntax to semantics.
Re:Claude was trained on protein sequences at scale
by zodwick on Wednesday November 18, 2026 @09:52AM
(#50547021)
good point on not needing labeling, hence proly makes sense to throw it in in pretraining and see if useful capabilities emerge. Funny how a lot of this is still a black box
Re:Claude was trained on protein sequences at scale
by dlougheed on Wednesday November 18, 2026 @06:28AM
(#50544975)
an LLM-predicted evolutionary tree seems monumentally useless, since it'll be unstable and you have no guarantees/metrics on any property or quality of the tree itself (parsimony, likelihood, estimated TMRCA, etc.)
Re:Claude was trained on protein sequences at scale
by zodwick on Wednesday November 18, 2026 @10:01AM
(#50547121)
out of my comfort zone, but out of curiousity; are these parameters ( parsimony, likelihood, estimated TMRCA, etc.) verifiable easily ? If so I wouldn't dismiss capabilities of LLMs in predicting these. All tasks that can we RL'd will be RL'd imho.
Re:Claude was trained on protein sequences at scale
by HackerThemAll on Wednesday November 18, 2026 @04:01AM
(#50543757)
No doubt that all of them, except maybe Gemini, surely were trained on SciHub and all books + data they could download from torrents and other sources, as well. That could include leaked personal data and proprietary copyrighted material, from anywhere maybe even including Darknet. Add to that public patent registry and the book buyout and scanning campaigns that occur.
I am certain that all these companies have stored and backed up all they could find, so that it can't be taken away from them.
Re:Claude was trained on protein sequences at scale
by arbot360 on Wednesday November 18, 2026 @05:45AM
(#50544589)
Goes to show how far Anthropic is going with unconventional RL tasks / synthetic datasets compared to others. The isn't really any natural human task that would require you to do this. This makes sense for their philosophy of sublimating everything into the model though, anything that can be done on a computer should be able to be approximated inside the model eventually.
Re:Claude was trained on protein sequences at scale
by yesornoisyes on Wednesday November 18, 2026 @09:48AM
(#50546976)
I wonder how they would do on novel protein sequences?
I mean, that’s the holy grail, no? In fact, I’m genuinely surprised the other labs don’t train on protein sequences. It’s both an excellent source of high quality information and potential breakthroughs.
Re:Claude was trained on protein sequences at scale
by spwa4 on Wednesday November 18, 2026 @06:16AM
(#50544869)
DNA is a language, or at least a protocol. Just a very difficult one. All meaning is locked up in the shape of the Lennard-Jones 12-6 magnetic flux isosurface that is generated when you string the letters together from amino-acids. Probably justified to call that a bit more indirect than English.
And electrostatic interactions.
And some methylation.
And some phosphorisation (but now we're getting deep into somewhat rare grammar)
Yeah, DNA is a language that is way too complicated for human minds to understand, although plenty of humans "get the gist" so to speak after looking at it for a few hours, but ... here's some bad news: it's only somewhat too complicated for LLMs. So why wouldn't you train LLMs on them? Frankly, it's the one language we have "texts" from that is actually difficult to understand for LLMs and ... well, it's definitely improving their understanding.
(also we have quite a bit of DNA text that isn't from the internet or books, it's freely downloadable and unlike everything else LLM companies steal, actually not covered by copyright)
Re:Claude was trained on protein sequences at scale
by pcrh on Wednesday November 18, 2026 @07:12AM
(#50545341)
It's hard to see what the point of this is?
When would one want to predict a full-length unknown protein sequence from only a fragment? Any fragment of any useful length can be back-translated to the DNA sequence and that can be tested against genomic databases (a process called tblastn). And if your protein/gene is not found there, you have no way of knowing whether or not the predicted sequence has any basis in reality.
Testing the model against shuffled sequences is also kinda cheating, as these are no longer random pool of amino acids, but one whose source already exists in the databases.
Re:Claude was trained on protein sequences at scale
by tstactplsignore on Wednesday November 18, 2026 @07:45AM
(#50545648)
I think the point is probably to improve reasoning about protein design work. If Claude has an innate understanding of protein sequence variation then it is able to just eyeball potential issues in protein designs. To be clear, I'm pretty sure Anthropic would ask claude to design proteins using task specific with conventional tools, but they then want it to be able to look at results and reason about them, akin to an expert. Not sure whether it'll actually help but I think that is the logic.
Re:Claude was trained on protein sequences at scale
by t0mek on Wednesday November 18, 2026 @09:15AM
(#50546583)
In "Empire of AI", the author mentions that when OpenAI included public GitHub repositories in its training data, the resulting model achieved better results even in fields unrelated to programming.
Aside from the obvious applications in biochemistry, could protein sequences also be used as a new training data source to improve overall model quality? If that's true, it's quite fascinating that we can take highly complex data we don't fully understand and use it to improve LLM training, simply because it contains so many significant structures and relationships.
Re:Claude was trained on protein sequences at scale
by phkahler on Wednesday November 18, 2026 @09:36AM
(#50546815)
Software is used in all fields, and github issues and PRs are going to contain detailed reasoning about all sorts of things when adding features or fixing bugs.
Re:Claude was trained on protein sequences at scale
by acka on Thursday November 19, 2026 @07:00PM
(#50568350)
It is important to note that there was a time when private repositories were not available to users on the free GitHub plan. For those wanting to use GitHub as their personal hard drive, this meant that the repositories containing their documents needed to be public.
Re:Claude was trained on protein sequences at scale
by dekhn on Wednesday November 18, 2026 @02:35PM
(#50550986)
It seems not entirely unreasonable to expect that training on large biological sequence repositories could uncover structure that is applicable to information theory. There has been a great deal of cross-pollination between those two fields.
Re:Claude was trained on protein sequences at scale
by jfrbfbreudh on Wednesday November 18, 2026 @09:56AM
(#50547062)
Given that there are quite a few quants who have switched over from HFT to AI research, it’s not surprising that AI models would be trained on unconventional / unexpected data sources. The fact that they included this training in their general release model shows that their internal experiments demonstrated that it was better including it than not.
But Anthropic has already started operating their wet lab, so it’s also entirely possible that Fable 5.2 is a distilled version of an internal model (though I doubt they would actually be that reckless).
Re:Claude was trained on protein sequences at scale
by highfrequency on Wednesday November 18, 2026 @10:07AM
(#50547206)
It would be cool if Claude were not explicitly trained on protein sequences at scale, and this were an emergent property from better fundamental understanding of chemical structure.
Re:Claude was trained on protein sequences at scale
by svnt on Wednesday November 18, 2026 @10:26AM
(#50547448)
In Nov/Dec 2025, just prior to the training cutoff for the first model to show this capability (Opus 4.7) I was working on solving arc-agi using claude code and was exploring using protein data and other adjacent data to improve performance.
In January it seemed very clear the subscription models behind claude code became less competent, there was a whole series of posts about it. I stopped working on it as a result.
Re:Claude was trained on protein sequences at scale
by mlmonkey on Wednesday November 18, 2026 @11:31AM
(#50548346)
Does shuffling still result in a viable protein? I'm no biochemist, so this is a naive question. Do proteins have some (chemical) constraints on how the "letters" can be arranged?
Re:Claude was trained on protein sequences at scale
by dguest on Wednesday November 18, 2026 @11:49AM
(#50548606)
Shuffling still gives you a viable amino acid sequence but it might fold up into garbage or just fall apart.
Real proteins have a number of structures which are composed from sequences and which make up the larger functional proteins. In the extreme case if you have a good idea what the protein does and an unprecedented way to predict its folding you could probably guess most of the sequence having never seen it.
You can think of it as computer code. It's a terribly written language that does no static checking or error handling, and in which every conceivable hack is used. But since many proteins do similar things they also reuse the same substructure.
Re:Claude was trained on protein sequences at scale
by mlmonkey on Wednesday November 18, 2026 @12:23PM
(#50549146)
[Displayingmy ignorance here] So is it possible that the model has learnt which letters are valid continuations? Like, for example, the rule in English "I before E except after C". Could the model just have figured out some simple rules from a small number of examples and is just applying them?
Re:Claude was trained on protein sequences at scale
by dguest on Thursday November 19, 2026 @05:03AM
(#50558140)
Disclaimer, I'm not a biologist, I just read wikipedia a lot.
But yeah proteins have several levels of structure [1]. The "primary" structure is the sequence, but there's a "secondary" within this sequence that codes for sub-sequences which form helices or sheets, which in turn glob together in the tertiary structure.
Re:Claude was trained on protein sequences at scale
by LarsDu88 on Wednesday November 18, 2026 @01:50PM
(#50550423)
I just tried this, and the ability of Opus to recognize functional residues from arbritrary protein sequence is quite extraordinary. A skill I don't think any humans have outside of focused understanding of specific proteins.
It already exists
It was included in the subscription up to monday (for mythos/fable).
It seems to be counting against my Fable limits.
On the other hand, selling on razor thin margins is one way to guarantee insufficient resources to verify real identities and monitor safety.
It's easy to trigger these rules - just ask anything that could even remotely indicate you're working on a biological weapon (or weapons in general). It's also very prone to happening when you do anything cybersecurity-related. I'm honestly surprised I managed to have Fable help me dig through Wireshark packet captures but it delegated most of the work to Opus sub-agents anyway so maybe that's what did the trick.
I'd love to do this. I'd love to know exactly which ADHD correlated mutations I have, but who the hell can I trust? Who can I trust to even run such a sequence instead of just half assing it or similar?
As for privacy: I didn't. I uploaded my genome to PGP (Personal Genome Project)- I just noticed their website doesn't seem to work anymore, and a few minutes of Googling suggest it is no longer easy to obtain (I kept a copy). I also posted a link to HN (12 years ago) which makes it pretty easy to identify me. I weighed the facts about genomes, identity, and health, and concluded that attempting to hide my genomics sequence was not an effective strategy.
As for you doing it, I honestly don't know. If your goal is to understand your disease (or diagnose yourself), genomic data is unlikely to provide you with anything accurate and/or actionable. Sorry I can't be more optimistic- with some notable exceptions, genomic data is really not that useful for health insights.
My own life science experience is that Fable 5 was completely unusable at first but 5.1, 5.2, and Opus 5.5 rarely trigger the safeguards.
I think I'm naive sometimes when it comes to trade-offs between freedom and security, but even I'm confused at why it would be a good thing for the whole of humanity having access to this stuff...
My only argument in favour is that it seems rather certain that at some point someone will leak the weights of a model with these capabilities so it might be safer if we brought forward the risks while they are still management so people take them extremely seriously.
If we go 5-10 years and then the weights of an ultra-capable model is leaked that may be game over.
What's the steelman argument for giving this research to Anthropic? Or any research that they are now gating?
> My only argument in favour is that it seems rather certain that at some point someone will leak the weights of a model
Yes. Because most of this research is either public, or should be public (but hidden behind bullshit fees by gatekeepers like Elsevier)
We can "protect" ourselves by limiting who has access, but:
- how often do those "trustworthy" elites use their abilities for abuse?
- it might leak
- someone else could independently obtain the capabilities
- an "act of god" might hit us without human intent
In any event, we're less prepared, because we assumed it couldn't happen. And it's not just bio.
And that's ignoring deeper questions of enforcement or censorship or surveillance.
I won't fight for a world where people are intentionally kept dumb, where the powerful are intentionally kept powerful, and where we intentionally keep ourselves vulnerable.
OpenClaude7 provides a sequence of DNA that in theory should be able to target and kill only black people (ignore the feasibility of this, it doesn't matter)
Bad guy now has a bunch of text and zero deployable pathogens to kill all the black people.
What do you think goes next in the list of steps? Where does the dying happen? How does Bad Guy turn theory into practice?
> Shuffling keeps the letters and destroys their order, so the difference in accuracy measures what the model knows about the sequence itself.
So by analogy, if you know a word in English starts with 'Q', you could reasonably predict the next letter would be 'u'. By shuffling, you might encounter the letter Q later on, and not successfully predict a 'u'.
Put another way, the shuffled sequences can only use frequency analysis for prediction, while the unshuffled ones have local and global order signals.
For a natural sequence, the observed molecule in nature is the source of truth.
However I thought the point was that the _difference_ between the shuffled prediction and the unshuffled is the measure here. So shuffled is the baseline, and the gap above that is how good the prediction is.
But the shuffled sequences aren't real sequences!
Sorry if I'm being daft here, but I still don't understand what the source of truth is for the shuffled sequence tests.
I wish the article had stated this clearly. It seems to imply the opposite, which is of course false.
I used to do bioinformatics, although more of the structure side than sequence. Like an information or understanding, it depends on what outcome you want that might benefit from the skill.
I suppose that predicting the next amino acid in a sequence would be useful for, say, reconstructing a protein sequence from fragments. Or constructing evolutionary trees of sequences by predicting the common ancestors?
Linking to ML space, is this analogous to knowing the model's weights byheart ? Traditionally this has been not useful to humans and machines, but recently some LLMs have started showing capability to directly edit the model weights to alter the behavious of smaller models. Is that a completely wrong analogy ?
Also I guess what I am partly curious is on how this data ended up in the model. Was it part of the pretraining corpus or was there an RL environment which specifically rewarded this behaviour
How useful the predicted sequence is for the predicted structure - and predicted function - no idea! There are already several layers of prediction here.
Sorry, I definitely do not know enough about LLMs to understand the analogy to weights. You can predict the effects of mutations on sequences, I think? As in, if I change this Q to an E will it make the enzyme faster (??). Perhaps you could LLM-generate a plausible member of a family of enzymes with a particular function, given fragments of an existing protein (???).
No idea how the data got in the model. It seems like the study that Xinming and Chen have done was designed to eliminate sources from the general corpus of knowledge - like any old science papers - b choosing obscure and rare sequences that might only be found in a sequence db.
jokes apart I do hope the LLMs ability to being a master at multiple unrelated fields unlock valuable insights
On the data getting in the model - I suspect: as the models have gotten bigger, the labs have started including more info from sequence databases of some sort
I am certain that all these companies have stored and backed up all they could find, so that it can't be taken away from them.
I mean, that’s the holy grail, no? In fact, I’m genuinely surprised the other labs don’t train on protein sequences. It’s both an excellent source of high quality information and potential breakthroughs.
And electrostatic interactions.
And some methylation.
And some phosphorisation (but now we're getting deep into somewhat rare grammar)
Yeah, DNA is a language that is way too complicated for human minds to understand, although plenty of humans "get the gist" so to speak after looking at it for a few hours, but ... here's some bad news: it's only somewhat too complicated for LLMs. So why wouldn't you train LLMs on them? Frankly, it's the one language we have "texts" from that is actually difficult to understand for LLMs and ... well, it's definitely improving their understanding.
(also we have quite a bit of DNA text that isn't from the internet or books, it's freely downloadable and unlike everything else LLM companies steal, actually not covered by copyright)
LJ potential is, but it doesn't have a magnetic flux or an isosurface.
When would one want to predict a full-length unknown protein sequence from only a fragment? Any fragment of any useful length can be back-translated to the DNA sequence and that can be tested against genomic databases (a process called tblastn). And if your protein/gene is not found there, you have no way of knowing whether or not the predicted sequence has any basis in reality.
Testing the model against shuffled sequences is also kinda cheating, as these are no longer random pool of amino acids, but one whose source already exists in the databases.
[0] https://assets.claude.com/30bf50e22a01388bb29bf077ee3f244531...
Doesn't anyone remember the Borderlands Science! mini-game that was in Borderlands 3?
https://www.nature.com/articles/s41587-024-02175-6
Aside from the obvious applications in biochemistry, could protein sequences also be used as a new training data source to improve overall model quality? If that's true, it's quite fascinating that we can take highly complex data we don't fully understand and use it to improve LLM training, simply because it contains so many significant structures and relationships.
> GitHub repositories ... in fields unrelated to programming
About as surprising as training on books and improving in fields other than writing.
We have seen that LLMs are quite good in reading binary and changing binary directly.
Could be possible.
But Anthropic has already started operating their wet lab, so it’s also entirely possible that Fable 5.2 is a distilled version of an internal model (though I doubt they would actually be that reckless).
In January it seemed very clear the subscription models behind claude code became less competent, there was a whole series of posts about it. I stopped working on it as a result.
Real proteins have a number of structures which are composed from sequences and which make up the larger functional proteins. In the extreme case if you have a good idea what the protein does and an unprecedented way to predict its folding you could probably guess most of the sequence having never seen it.
You can think of it as computer code. It's a terribly written language that does no static checking or error handling, and in which every conceivable hack is used. But since many proteins do similar things they also reuse the same substructure.
But yeah proteins have several levels of structure [1]. The "primary" structure is the sequence, but there's a "secondary" within this sequence that codes for sub-sequences which form helices or sheets, which in turn glob together in the tertiary structure.
So there's a rich "grammar" to them.
[1]: https://en.wikipedia.org/wiki/Protein_structure