News for hackers. Stuff that matters.
Re:An Introduction to Compassionate Screen Scraping by meatmanek on Tuesday April 19, 2011 @05:05PM (#2464226)
The ability to use CSS/jQuery selectors is really nice, though; In order to find all <td>s whose parents have a class "blah", you have to use list comprehension:

  tds = [td for td in soup.findAll('td') if td.parent.get('class') == 'blah']
In jQuery, this is more compactly written:

  tds = $('.blah > td')
And if you just want to look for <td>s somewhere within a .blah element, you can use

  tds = $('.blah td')
This is a lot less clear in BeautifulSoup:

  tds = [td for td in soup.findAll('td') if td.findParents(attrs={'class': 'blah'})]

(If there are better ways to write this BeautifulSoup code, please let me know)

Selectors have some other benefits too - you can just go to the CSS file and grab the selector that matches what you want, and you can be reasonably sure it'll work in most cases.

Re:An Introduction to Compassionate Screen Scraping by joshu on Wednesday April 20, 2011 @03:13AM (#2465614)
Lxml in python lets you use CSS selectors.