No, DeepMind has not solved protein folding

Re:No, DeepMind has not solved protein folding by dnautics on Friday December 04, 2020 @03:40PM (#25307596)

> for example in NLP, many people assume a high log prob score means a high confidence but it is not true at all

Thank you for the clarification. I wasn't aware of this, as I'm most familiar with super-basic/old NLP techniques like BOW/RNN/LSTM/GRU techniques, where log prob scores seemed to me to be roughly correlated with result quality. I'm aware the landscape has changed recently with insights about high dimensional searches...

[ Parent | on HN ]

Re:No, DeepMind has not solved protein folding by visarga on Saturday December 05, 2020 @02:47AM (#25312690)

There is no difference in the log prob between LSTM and transformers. It just depends on the number of updates you make (batch size, epochs), if you train it to overfit, then your log probs will be pushed closer to 1 or 0.

But after training you can recalibrate the temperature of the softmax on the test set and still get meaningful confidence scores (temperature calibration). Or you can use a variation of cross entropy called Focal Loss that will leave your logits un-squashed.

[ Parent | on HN ]

Re:No, DeepMind has not solved protein folding by Voloskaya on Friday December 04, 2020 @03:48PM (#25307711)

An interesting paper on this subject from just last week if you want to know a bit more:

http://phontron.com/paper/jiang20lmcalibration.pdf

[ Parent | on HN ]