Re:The Verge's web sucks
by dvh on Friday July 24, 2015 @03:46AM
(#9940614)
After I started working on my own RSS reader, I started converting more and more website to RSS feeds (e.g. twitter, youtube, all news site) where I only extract interesting bits. It's faster and cleaner, not to mension no ads.
[ Parent
| on HN ]
More and more sites (twitter is a major offender here) are switching off their RSS feeds because they can't control the readers.
See for instance:
https://api.twitter.com/1/statuses/user_timeline.rss?screen_...
So I'm back to what I used to do 13 years ago - build my own little scrapers and adaptors to generate RSS feeds for myself :) WHEEEEEE!
Would you mind sharing the program?
There are plenty of readers, you just need to create the html to rss convertor for specific sites. My code is quite ugly so no. Also they often break when they make changes on website.
The converter is actually what I would really like to have. I hate parsing. And fixing it when it breaks. It doesn't matter if the code is ugly. You are trying to parse crappy websites after all. Consider sharing.
Well, I only have parsers for slovak news site, 3 parsers for lang-8 (homepage, lang match, footprints), couple for local real estate agents, nothing reusable for english speaking folks, only thing that you could use is probably twitter parser: http://pastebin.com/wpjZ6azZ I did it just recently and I use it to parse DAWN and New Horizont feeds.