Re:Full Text RSS Feed: Get the whole feed and nothing but th...
by benmccann on Tuesday March 01, 2011 @11:09PM
(#2278407)
I don't think it's quite as good as what he's doing though. He has the title and date specifically pulled out and he doesn't have any extra text included. I think he manually handles CNN. If I try a HuffPost feed it doesn't work at all.
[ Parent
| on HN ]
Re:Full Text RSS Feed: Get the whole feed and nothing but th...
by yesimahuman on Wednesday March 02, 2011 @12:47AM
(#2278600)
Yea, I'd be curious to see exactly what he's doing. I can only guess there is a heuristic which results in a lot of failed feed processing noticed on here (I know it's just a weekend project :)) that doesn't generalize well. Boilerpipe, in my experience, works very well on almost all news/blog type content. Finding the date in the first few sentences and the title are extra heuristics that can be added later.
EDIT: The date and title are in the RSS feed already! No further analysis needed.