News for hackers. Stuff that matters.
Re:DeepMind uses the Daily Mail as a huge training corpus fo... by paulsutter on Tuesday June 23, 2015 @01:28AM (#9762782)
The original Deepmind paper [1] is based on a really smart idea. Algorithmic development relies on measuring the performance of any proposed algorithm. For reading comprehension, performance is evaluated using Q&A about the corpus. Its difficult to find a large corpus with a comprehensive set of questions about the content.

Deepmind is cleverly converting the Daily Mail article summaries into questions by removing a proper noun. For example:

Question: Producer X will not press charges against Jeremy Clarkson, his lawyer says.

Answer: Oisin Tymon

They are using the Daily Mail corpus to develop their algorithm, and that's smart. They aren't relying on it as an important source of information. Maybe all you guys with the dismissive comments have a better idea?

[1] http://arxiv.org/pdf/1506.03340.pdf

EDIT: Thanks Otik, reworded the opening sentance

Re:DeepMind uses the Daily Mail as a huge training corpus fo... by Otik on Tuesday June 23, 2015 @03:08AM (#9763055)
For a moment I thought you were calling the Mail "a great paper", but actually the simple language used by it is probably quite good for this purpose.

Of course, it does mean that if you ask Deep Mind about the cause of anything negative it will probably tell you "immigrants".

Re:DeepMind uses the Daily Mail as a huge training corpus fo... by collyw on Tuesday June 23, 2015 @03:57AM (#9763206)
or "benefits scroungers"
Re:DeepMind uses the Daily Mail as a huge training corpus fo... by PlzSnow on Tuesday June 23, 2015 @04:57AM (#9763413)
edgy stuff
Re:DeepMind uses the Daily Mail as a huge training corpus fo... by mcintyre1994 on Tuesday June 23, 2015 @01:42AM (#9762811)
I agree with you - it's a neat idea, though I haven't read the paper yet.

I wonder if it has explicit knowledge that they'll always remove a proper noun. Eventually the daily mail will write some article about a place causing or curing cancer, and the poor thing will break down trying to find X in X causes cancer from that corpus :)

On another note, I wonder if they could train it on Buzzfeed-esque content without changing the headlines. "You won't believe who Jeremy Clarkson punched!" -> the answer

Re:DeepMind uses the Daily Mail as a huge training corpus fo... by tormeh on Tuesday June 23, 2015 @07:54AM (#9764018)
The daily mail should not be a source of information at all, though. Any greater weighting than zero is too much. If it's only used as a test for the algorithm then I guess it can be OK, assuming the Mail's broken logic doesn't get in the way. It's the step above dream logic, really. Deepmind might have problems abstracting too much over the text because of this. Trying to combine information from several different articles would be basically impossible.

Anyway, how about using Wikipedia instead? I can only guess they're using the Mail because they think it's funny.

Re:DeepMind uses the Daily Mail as a huge training corpus fo... by erroneousfunk on Tuesday June 23, 2015 @12:24PM (#9765705)
"I can only guess they're using the Mail because they think it's funny."

Ironically, given your self-professed superior tastes in reading material, Deepmind seems to have a better ability to read, comprehend, and answer questions about articles than you do.

Re:DeepMind uses the Daily Mail as a huge training corpus fo... by peteretep on Tuesday June 23, 2015 @02:26AM (#9762938)
So they're using a corpus as a determining factor for quality assessments, but you don't think that makes it an important source of information?
Re:DeepMind uses the Daily Mail as a huge training corpus fo... by wodenokoto on Tuesday June 23, 2015 @03:26AM (#9763110)
Thanks for the summary. I really didn't understand what the article was trying to say they where doing.