News for hackers. Stuff that matters.
Re:AlphaFold: a solution to a 50-year-old grand challenge in... by sampo on Monday November 30, 2020 @06:27PM (#25260393)
> The other major case in point being NLP

Speaking of which, Google Translate was published in 2006, but when did the "learning from data" approach became an accepted idea in machine translation? I think the earlier attempts at machine translation were more about trying to codify grammar rules in software, than doing statistical learning from large text corpuses? I remember in 2002, the approach of leaning protein substructures from data was already the best performing approach in the protein folding problem.

Re:AlphaFold: a solution to a 50-year-old grand challenge in... by Polygator on Monday November 30, 2020 @10:51PM (#25262105)
Not really. Using a statistical approach to text modelling, specifically using Markov Chains, was proposed by Shannon in 1948. But yeah, there's a point in the 2000s where generative grammar/ symbolic approaches were pretty much left behind by NN methods.

When we discuss Google's input in NLP, the most important contribution is certainly the "Attention is all you Need" paper, which paved the way for BERT and GTP (Alphafold also uses Attention networks, btw)

Re:AlphaFold: a solution to a 50-year-old grand challenge in... by adw on Monday November 30, 2020 @11:34PM (#25262332)
> generative grammar/ symbolic approaches were pretty much left behind by NN methods

Which is the same thing as hand-engineered feature stacks being left behind in vision problems, really. The story in every field is more or less "you're not clever enough to engineer good features"; "you might be clever enough to define good symmetries for the feature space in which the features live... maybe" (convolutional neural networks in image problems); "... but maybe not even that" (attention mechanisms).

Re:AlphaFold: a solution to a 50-year-old grand challenge in... by snovv_crash on Tuesday December 01, 2020 @01:31PM (#25268172)
The hand generated features are still superior for SfM style problems, where the geometry is well defined but would need to be learned by the NN from scratch.
Re:AlphaFold: a solution to a 50-year-old grand challenge in... by std_badalloc on Tuesday December 01, 2020 @03:34AM (#25263525)
I think it was with the seq2seq paper of Sutskever, Vinyals and Le in 2014: https://arxiv.org/abs/1409.3215

People were doing a mix of learning from data and hand engineered solutions before this, but this was the first system learned end-to-end, afaik.

Re:AlphaFold: a solution to a 50-year-old grand challenge in... by xbmcuser on Monday November 30, 2020 @09:23PM (#25261578)
I think Netflix model of simplifying then translating should work better with internet forums and blog posts. Looking at how google works I was hoping someone in google wold adopt it and release a competing product against google translate https://arxiv.org/abs/2005.11197