Re:ScraperWiki - an online tool to make scraping simpler and...
by pyre on Sunday August 08, 2010 @05:03AM
(#1585091)
How relevant are these posts now?
http://blog.ianbicking.org/2008/12/10/lxml-an-underappreciat...
http://blog.ianbicking.org/2008/03/30/python-html-parser-per...
Re:ScraperWiki - an online tool to make scraping simpler and...
by iamelgringo on Sunday August 08, 2010 @11:49AM
(#1585475)
Very.
I recently switched from BeautifulSoup to lxml. It's tons faster, and I like it quite a bit more than BeautifulSoup. The documentation isn't quite as good, and it's a bit of a pain to install, but if you have to do a bunch of scraping, I'd check it out.
Re:ScraperWiki - an online tool to make scraping simpler and...
by evilduck on Sunday August 08, 2010 @01:47PM
(#1585724)
What pain? I've only used lxml for a project on Windows, but there, it was as simple as installing setuptools and then 'easy-install lxml'.
Re:ScraperWiki - an online tool to make scraping simpler and...
by pyre on Sunday August 08, 2010 @05:35PM
(#1586228)
I think the 'pain' comes when there are issues with the c library that lxml binds to.
Re:ScraperWiki - an online tool to make scraping simpler and...
by andrix on Monday August 09, 2010 @07:55AM
(#1587689)
Python has Scrapy (http://scrapy.org), quoting:
"Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing."
Re:ScraperWiki - an online tool to make scraping simpler and...
by jhy on Sunday August 08, 2010 @10:09PM
(#1586786)
I've been working on a Java HTML parser:
http://jsoup.org/
It supports CSS selectors, jquery-like DOM traversal and set manipulation, etc.
Re:ScraperWiki - an online tool to make scraping simpler and...
by evilduck on Sunday August 08, 2010 @01:44PM
(#1585720)
I was under the impression BeautifulSoup is no longer actively maintained.
(Edit: Which makes me sad, because I loved their documentation writing style)
Re:ScraperWiki - an online tool to make scraping simpler and...
by krakensden on Sunday August 08, 2010 @10:55PM
(#1586873)
It's not :(
Re:ScraperWiki - an online tool to make scraping simpler and...
by astrofinch on Sunday August 08, 2010 @06:54PM
(#1586411)
No idea.
http://www.crummy.com/software/BeautifulSoup/