News for hackers. Stuff that matters.
Re:Show HN: TLDR This – Auto summarize any article or webpag... by bhl on Sunday March 15, 2020 @02:58PM (#22585716)
You can use tf-idf [1] to achieve step 2 and that extra fancy part of excluding commmon articles and prepositions: count the frequency of words in the article, but divide it by the sum of frequencies from past articles.

Text summarization works as a good toy problem, because it leads to two harder problems: 1. text extraction (how to distinguish content from non-content like ads) 2. q&a (given text and a question about the text, how can you produce an answer).

[1] https://en.wikipedia.org/wiki/Tf%E2%80%93idf

There are no comments on this comment yet.