Re:Show HN: Kimono – Never write a web scraper again
by pranade on Wednesday January 15, 2014 @10:15PM
(#7067850)
Kimono can handle several pages with malformed and old/bad HTML. We're still in beta though, so we're handling more edge cases as we encounter them. Should work on HN main page.
[ Parent
| on HN ]
Re:Show HN: Kimono – Never write a web scraper again
by tlarkworthy on Thursday January 16, 2014 @08:13AM
(#7069162)
I developed some hot strategies for the next level ...
http://edinburghhacklab.com/2013/09/probabalistic-scraping-o...
Re:Show HN: Kimono – Never write a web scraper again
by Nilzor on Friday January 17, 2014 @02:13AM
(#7074680)
Have you thought of implementing a system for custom handling of edge cases? Realistically you can't handle all edge cases, so leaving the last 2% up to the user (which is a developer) would be a good idea. It'd still save him 98% of the work, but also give him the comfort of knowing it won't break even if site X is doing something weird with their HTML in the future.
I'll leave it to you to figure out how one would implement such a feature ;-)