News for hackers. Stuff that matters.
Re:Alphafold by timr on Thursday July 15, 2021 @03:22PM (#27848935)
> A disturbing thing is that the architecture is much less novel than I originally thought it would be, so this shows perhaps one of the major difficulties was having the resources to try different things on a massive set of multiple alignments. This is something an industrial lab like DeepMind excels at. Whereas universities tend to suck at anything that requires a directed effort of more than a handful of people.

Yeah, the HN commentary on Alphafold has a high heat-to-light ratio. I'm eager to read the paper because the previous description of the method sounded remarkably similar to methods that have been around for ages, plus a few twists.

The devil is going to be in the details on this one.

Re:Alphafold by MrsPeaches on Thursday July 15, 2021 @03:55PM (#27849341)
> high heat-to-light ratio

Sorry for the ignorance but what does this mean?

Re:Alphafold by iand on Thursday July 15, 2021 @07:27PM (#27851283)
It means that the conversation isn't producing much of value despite lots of activity, like an inefficient lightbulb that wastes energy by emitting heat instead of light.
Re:Alphafold by timr on Thursday July 15, 2021 @04:04PM (#27849444)
It's an idiom implying that there's a lot of chatter and bold claims, but very little of it is factual or informative.
Re:Alphafold by Azrael3000 on Thursday July 15, 2021 @03:59PM (#27849393)
Incandescent light bulbs are generally very inefficient in producing light, compared to LED for example. They produce a lot of heat and not much light for which they are made.

So in this context I suppose that gp implies that these threads don't provide much meaningful discussion but rather lots of hand waving.

Re:Alphafold by HPsquared on Thursday July 15, 2021 @05:40PM (#27850459)
Light is also often used in metaphors relating to knowledge, wisdom etc.
Re:Alphafold by dekhn on Thursday July 15, 2021 @06:03PM (#27850648)
"Fiat Lux" not "Fiat Calor"
Re:Alphafold by AlexCoventry on Thursday July 15, 2021 @05:47PM (#27850522)
Emotion-to-understanding ratio
Re:Alphafold by remexre on Thursday July 15, 2021 @07:18PM (#27851221)
Heat = Flaming, Light = Illumination (in the metaphorical sense)
Re:Alphafold by butMuhCulture on Thursday July 15, 2021 @03:59PM (#27849384)
It’s trying to say light is more valuable than heat, or some such folksy thing. I cook steak in the dark so I don’t find it to be a very insightful metaphor.
Re:Alphafold by varelse on Saturday July 17, 2021 @01:51PM (#27867481)
Transformers seem to be the successor to conv nets. But in my direct experience advocating for them, it's amazing how reluctant industry peeps were to trying them because they associated all the limitations of LSTM networks with them for a long time. YMMV, but that's how it went with me.

I even predicted DeepMind's CASP 14 network would be transformer-based back in 2018, but I couldn't have told you the details of that transformer, just that it was a no-brainer to move from fixed width convolutions to arbitrary width attention sums because sequence motifs and long-range interactions are of arbitrary width in the sequence.

All that seems to have changed with AlphaFold 2 because unlike GPT-XXX, this isn't a parlor trick with memorized text. This is actually useful and the FOSSing of the network will spawn all sorts of new applications of the approach.

So now I wonder what will replace Transformers because nothing lasts forever and there are a lot of smart people trying all sorts of new ideas.

Re:Alphafold by dm319 on Thursday July 15, 2021 @04:46PM (#27849921)
The key difference seems to be using the multiple alignments and assumption about evolutionary conservation? Useful for genes conserved, but less useful for de-novo proteins (like COVID and cancer) I guess?
Re:Alphafold by timr on Thursday July 15, 2021 @05:13PM (#27850200)
Dunno yet. MSAs were always a key input to Rosetta (previous best method). How they were used was very different.

Fundamentally, everything in this space (= non-physical methods) is about inferring structure from things that are closely related. And you can't solve the problem at all for non-trivial proteins using physics, so here we are.

Re:Alphafold by feanaro on Friday July 16, 2021 @02:37AM (#27853575)
> And you can't solve the problem at all for non-trivial proteins using physics, so here we are.

I'd appreciate if you could expand a bit on what you meant here, sounded interesting.

Re:Alphafold by dm319 on Friday July 16, 2021 @12:28PM (#27857970)
I guess poster is referring to purely using physics to work out the structure - i.e. no more knowledge than how atoms/molecules move and the sequence. At the moment knowledge is gleaned from evolution by virtue of evolutionary conservancy.
Re:Alphafold by timr on Tuesday July 20, 2021 @01:46PM (#27896831)
Essentially yes. In theory, you could start from quantum mechanics and get the structure of any chemical matter. In practice, that's intractable. So there has been decades of work to make ever-simpler, less expensive approximations, and use these models of forces/energies to predict the structures of complex biomolecules.For example, google "molecular mechanics" or "molecular dynamics", and you'll find discussion o f things like the van der waals model of an atom, models of chemical bonds that resemble springs from physics 101, coulomb's law, and so on.

People take these simple models, chain them together in different ways, adjust the free parameters (i.e. weights) by fitting to known data, and thus create "force fields" that can be used to estimate the energies and/or forces on a molecule, given only the coordinates of the atoms. These methods work OK for some limited problems, but tend to fly off the rails as simulation times get longer, or if atomic systems have more electrostatics, motion, "weird" atoms (like heavy metals), etc.

This (Alphafold) kind of work is completely different, in that it starts from data, and uses the physical models only to do final refinements (if at all).

Re:Alphafold by TaupeRanger on Thursday July 15, 2021 @05:28PM (#27850363)
That's the case with basically everything DeepMind does. They have a very good PR department which hypes up everything they do while conveniently ignoring that basically nothing of any practical consequence has come of their endeavors. But I do think it's important that these companies exist now so we can see what not to try going forward.
Re:Alphafold by timr on Thursday July 15, 2021 @05:43PM (#27850491)
Well, the CASP14 results do speak for themselves. Protein structure prediction is not necessarily of great meaning to drug discovery or biology, but they pretty much blew everyone else out of the water in a fair contest. For that reason, they deserve praise.

It's a little like making a robot that is very, very good at something pointless (say, using a yo-yo). Who knows where it might lead, but if they make the best damned yo-yo bot in the world, they deserve whatever praise they get from the yo-yo community.

Re:Alphafold by Jyaif on Friday July 16, 2021 @06:26AM (#27854749)
Their PR strategy is to take problems people thought were impossible to solve in the next 10 years, and solve them (Go) or nearly solve them (StarCraft 2, protein solving)
Re:Alphafold by TaupeRanger on Sunday July 18, 2021 @10:41PM (#27878730)
No one thought that Go was impossible except people who weren't involved in engineering/software and shouldn't have been taken seriously in the first place.

Superhuman game AI has existed for decades. It's entirely unsurprising that one can play a strategy game, and no one thought it wasn't possible.

I'll send you $500 USD if AlphaFold derivatives lead to a single new therapy or drug that helps real patients in the next 5 years.

Re:Alphafold by sdenton4 on Friday July 23, 2021 @01:35PM (#27933420)
'In the next 10 years' is the key phrase here. Everyone watching Go (I looked at the problem 15-ish years ago) thought it required a massive advance to beat the best humans. DeepMind made that advance.

'bah, whatever, both Newton and Leibniz were hacks. Everyone knew calculus would have been invented anyway. It's basically a consequence of some stuff Archimedes did.'

Re:Alphafold by dekhn on Friday July 16, 2021 @09:21AM (#27855940)
DM didn't "solve" anything around proteins. They just made an improvement to existing homology modelling methods. If you look, the system is incredibly dependent on having large numbers of high quality sequence alignments to proteins with known structure and lots and lots of evolutionary data.

This was actually fairly obvious 20 years ago and it's been disappointing to me to see how long it took somebody to make this improvement, but really, it couldn't have been done without recent algorithmic improvements and huge amounts of CPU time.

Re:Alphafold by sdenton4 on Friday July 16, 2021 @04:39AM (#27854192)
Well here's one example: Deepmind made Wavenet, which turned text-to-speech on it's head. Variants of Wavenet underlie most or all of the talking machines (Google assistant, Alexa, etc.).
Re:Alphafold by TaupeRanger on Sunday July 18, 2021 @10:31PM (#27878675)
That's pretty much it. A maybe-slightly-better computer voice.
Re:Alphafold by sdenton4 on Friday July 23, 2021 @01:31PM (#27933385)
Well, also Alpha-Fold, and let's not forget beating the world champions at Go, which was a really major unsolved problem. I'll also contend that WaveNet was not 'maybe-slightly-better,' but actually a massive leap forward.

The phrase 'who pissed in your cheerios' comes to mind... Here's hoping that if you ever build something worthwhile it is received with a bit more compassion.