One crazy idea I have is to run some very crude ab-initio QM or DFT stuff starting with folded proteins, and gradually running the temperature higher until it unfolds. Then amass a dataset of protein structures + positional delta vectors. Then time-reverse the dataset (flip the sign on those vectors). Then train a 3d convolutional NN on the reverse-melting curve to obtain heuristic rules for folding in whatever universe the shitty physics engine represents.

Then it doesn't matter if the QM simulation is very crude and deeply flawed, so long as it gets to the right answer at the end.

IIUC the folding and unfolding pathways of a protein are not time reversed wrt each other.

But you would still enjoy reading this: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC17732/ the work in this paper led to folding@home because vijay couldn't get enough computer time to run his simulations

if you really had a lot of computer time to waste, you could imagine doing simulations where you titired in or out some guanadinum chloride and inspected how disrupting h-bonds (versus hydrophobic collapse) contributes. Chaotropes are better than temperature for probing unfolding.

> IIUC the folding and unfolding pathways of a protein are not time reversed wrt each other.

That's more or less correct, but in my proposal, we are not modeling the folding pathway of a protein. One way to think of it: we would be modelling the reverse unfolding pathway of a protein in a universe with different (simpler) physics than ours, physics in which (hopefully) the time reversed unfolding pathway yields the correctly folded protein in OUR universe.

Your description kind of reminds me of the 2018 World Models publication. Are you familiar with that work and/or am I off base? (https://worldmodels.github.io/) (https://arxiv.org/abs/1803.10122)

Off base. I'm using universes as a florid description of the fact that our QM models are crap but they might be useful of the laws of physics were different.

Check out DESMOND and ANTON, that's basically what DE Shaw Research is doing. It does seem like, to get static structure predictions, it's going to be hard for anybody to find anything that does marginally better than AF2 at this point, but since static structure predicitons are just mostly useful for brainstorming, I think ANTON may end up being more useful, in terms of applied science.

I wonder if a transformer with 1 recurrent layer (or a transformer Deep Equilibrium Model) could work well. Transformers are almost like a physics simulation in that they sum vector-valued interactions which depend on distance in some space, and then add the result to the state of each particle / element.

Reasonably speaking, I would expect that within 2 years 10 groups will be as proficient as AF2 at predicting static structures. I don't think anybody who is trying to emulate physics simulations will be in that group, just folks who have learned enough tricks to quickly incorporate all the evidence during training and choose how to apply it during prediction.

I expect to see a "multiple feature embedding heads on top of 2 fullly connected layers" (as used in modern ads training) will end up being the simplest architecture capable of folding proteins well.