Re:Ask HN: What is your most impressive hackathon project?
by assafmo on Sunday May 28, 2017 @04:51PM
(#14436036)
I would think it depends on the site... maybe some special html characteristics around the first headline?
I usually use lynx, which strips down the html and only outputs the text. News site tend to have the same structure for every article, so after passing the html to lynx, the headline for a news site article will probably be around line 5-6.
To second another comment, I am also working on scraping but for articles. In this particular example I am trying to figure out how to get the scraper to start on the first headline all the way down to the bottom, and not touch anything else. Any tips for how to get it to find the first heading on a page?