Re:Show HN: Kimono – Never write a web scraper again
by littledot5566 on Thursday January 16, 2014 @03:21AM
(#7068602)
Perhaps this could be automated by finding the same content in two versions of the dom and then doing a diff on the structure, updating the rules?
Re:Show HN: Kimono – Never write a web scraper again
by pranade on Thursday January 16, 2014 @12:29PM
(#7070862)
It would be great to automate this eventually. For now, we're trying to make it really easy to set up and rebuild the scraper. If it goes down, you'll see it in the status on your user dashboard. We're also implementing alerts, so you can opt to get an email notification if a scrape fails
I disagree. Web scraping is mostly fun. You don't need "a ton of code" and "a laundry list of libraries", just something like Beautiful Soup and maybe XSLT.
The end of the statement is truer: it's not really a problem that your web scraper will have to be hosted somewhere, since the thing you're using it for also has to be hosted somewhere, but yes, it needs to be maintained and it will break if the source changes.
But I don't see how this solution could ever be able to automatically evolve with the source, without the original developer doing anything?