News for hackers. Stuff that matters.
Re:ScraperWiki - an online tool to make scraping simpler and... by astrofinch on Sunday August 08, 2010 @03:32AM (#1585010)
Re:ScraperWiki - an online tool to make scraping simpler and... by pyre on Sunday August 08, 2010 @05:03AM (#1585091)
Re:ScraperWiki - an online tool to make scraping simpler and... by iamelgringo on Sunday August 08, 2010 @11:49AM (#1585475)
Very.

I recently switched from BeautifulSoup to lxml. It's tons faster, and I like it quite a bit more than BeautifulSoup. The documentation isn't quite as good, and it's a bit of a pain to install, but if you have to do a bunch of scraping, I'd check it out.

Re:ScraperWiki - an online tool to make scraping simpler and... by evilduck on Sunday August 08, 2010 @01:47PM (#1585724)
What pain? I've only used lxml for a project on Windows, but there, it was as simple as installing setuptools and then 'easy-install lxml'.
Re:ScraperWiki - an online tool to make scraping simpler and... by pyre on Sunday August 08, 2010 @05:35PM (#1586228)
I think the 'pain' comes when there are issues with the c library that lxml binds to.
Re:ScraperWiki - an online tool to make scraping simpler and... by andrix on Monday August 09, 2010 @07:55AM (#1587689)
Python has Scrapy (http://scrapy.org), quoting: "Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing."
Re:ScraperWiki - an online tool to make scraping simpler and... by jhy on Sunday August 08, 2010 @10:09PM (#1586786)
I've been working on a Java HTML parser: http://jsoup.org/

It supports CSS selectors, jquery-like DOM traversal and set manipulation, etc.

Re:ScraperWiki - an online tool to make scraping simpler and... by evilduck on Sunday August 08, 2010 @01:44PM (#1585720)
I was under the impression BeautifulSoup is no longer actively maintained.

(Edit: Which makes me sad, because I loved their documentation writing style)

Re:ScraperWiki - an online tool to make scraping simpler and... by krakensden on Sunday August 08, 2010 @10:55PM (#1586873)
It's not :(
Re:ScraperWiki - an online tool to make scraping simpler and... by astrofinch on Sunday August 08, 2010 @06:54PM (#1586411)
No idea.