Instapaper is joining Pinterest

Re:Instapaper is joining Pinterest by kough on Tuesday August 23, 2016 @02:21PM (#12346093)

Readability (https://github.com/luin/readability) is a classic, and included as part of Firefox (I think, maybe that's been discontinued). It's essentially a bag of hand-written heuristics but they're pretty good heuristics.

Some interesting reading is Christian Kohlschütter's thesis on this problem, which is framed in academia as "how do we assemble good text corpuses from webpages for data analysis, which means removing junk (boilerplate) from our HTML crawls" (https://code.google.com/archive/p/boilerpipe/wikis/WSDM2010P...). Boilerpipe would probably be the right way to go, but if you're not using Java it could be harder to integrate.

[ Parent | on HN ]

Re:Instapaper is joining Pinterest by userhacker on Tuesday August 23, 2016 @02:54PM (#12346428)

I second Readability, it works great for article heavy webpages. I used it to build a reading time estimator for chrome https://chrome.google.com/webstore/detail/read-time/nccohhim... and its open source https://github.com/usergit/read-time bonus, you can click on the extension to show only the main content of the page

[ Parent | on HN ]

Re:Instapaper is joining Pinterest by idlewords on Tuesday August 23, 2016 @03:26PM (#12346711)

Thank you and the parent both for these links!

[ Parent | on HN ]

Re:Instapaper is joining Pinterest by axx on Tuesday August 23, 2016 @03:53PM (#12346938)

It works great! I used the Pinboard API to download my bookmarks and used readability to crawl the original text from the source url.

[ Parent | on HN ]

Re:Instapaper is joining Pinterest by cpeterso on Tuesday August 23, 2016 @03:25PM (#12346699)

Firefox maintains a fork of Readability for Firefox's reader mode here:

https://github.com/mozilla/readability

[ Parent | on HN ]