Re:An Introduction to Compassionate Screen Scraping
by meatmanek on Tuesday April 19, 2011 @05:05PM
(#2464226)
The ability to use CSS/jQuery selectors is really nice, though; In order to find all <td>s whose parents have a class "blah", you have to use list comprehension:
tds = [td for td in soup.findAll('td') if td.parent.get('class') == 'blah']
In jQuery, this is more compactly written:
tds = $('.blah > td')
And if you just want to look for <td>s somewhere within a .blah element, you can use
tds = $('.blah td')
This is a lot less clear in BeautifulSoup:
tds = [td for td in soup.findAll('td') if td.findParents(attrs={'class': 'blah'})]
(If there are better ways to write this BeautifulSoup code, please let me know)
Selectors have some other benefits too - you can just go to the CSS file and grab the selector that matches what you want, and you can be reasonably sure it'll work in most cases.
Selectors have some other benefits too - you can just go to the CSS file and grab the selector that matches what you want, and you can be reasonably sure it'll work in most cases.