Re:Text Summarization with TensorFlow
by hisham_hm on Wednesday August 24, 2016 @06:42PM
(#12355805)
I don't see that as a failure. It did produce a sentence that is shorter, grammatical (though "named to head" is a bit weird) and essentially true — calling Google an "internet company" would make sense in its early days (back when Google would be prefixed by "internet search-engine company").
I didn't think it was a failure either until I realized that I was letting future knowledge leak into the past! There is more than one internet company so upon reading that headline, given that it must be a novel event, my question would be: "Which Company?". Now I have to read the article until I find out. The human summary is better because I don't have to ask that question.
Most humans would interpret "novell chief executive named to head internet company" to mean "novell is an internet company and its chief executive just became its head" which is incorrect (and a little nonsensical since the CEO already is in change).
That's pretty interesting. It's taken me 5 readings of that sentence, including once out-loud to get your reading of it.
I thought the generated summary was really, really good. But I knew that Novell wasn't considered an internet company, so it wasn't until I made myself ignore that before I could see the other reading.
Oh, that's funny. I was thinking it was. I guess it's a software company but not an internet company.
I think the holiday pay example is more glaring. Seen in isolation, I would be confused as to what on earth it was getting at. Furthermore, the summary is no good. The abstract isn't either. My summary would be: British Gas continues to fight eu court's decision that commission be included in holiday pay. Continue is used here to emphasize that the case is not yet over.
On the other hand, the football summary is exemplary; better than the provided abstract.
article: novell inc. chief executive officer eric schmidt has been named chairman of the internet search-engine company google .
human: novell ceo named google chairman
machine: novell chief executive named to head internet company
https://github.com/tensorflow/models/tree/master/textsum
(It's still great that it beats all competition on the ROUGE score, of course.)