News for hackers. Stuff that matters.
Re:Scrapy Tips from the Pros by bbayer on Wednesday January 20, 2016 @02:43PM (#10940769)
I love scraping web and produce structured data from web pages. The only downside of using XPath or similar extracting approach is necessity of constant maintenance. If I have enough knowledge about machine learning, I would like to write a framework that analysis similar pages and finds structure of data without giving which parts of page should be extracted.
Re:Scrapy Tips from the Pros by stummjr on Wednesday January 20, 2016 @03:11PM (#10940966)
Maybe you should give a try at Portia (http://scrapinghub.com/portia/). It does exactly what you mean.

You may also be interested in this library: https://github.com/scrapy/scrapely