Re:Web scraping via JavaScript runtime heap snapshots
by goldenkey on Friday April 29, 2022 @04:39PM
(#31210324)
CSS selectors don't have to select purely by classes. They can be something like:
div > div > * > *:nth-child(7)
XPath doesn't have any additional abilities, it's just verbose and difficult to write. It's a lemon.
Re:Web scraping via JavaScript runtime heap snapshots
by tommica on Friday April 29, 2022 @05:30PM
(#31210788)
I might be wrong, but xpath has contains, where you can look for a text content inside an element, which I don't think CSS can do
Re:Web scraping via JavaScript runtime heap snapshots
by mdaniel on Friday April 29, 2022 @05:59PM
(#31211047)
Yeah, for sure XPath is the more powerful of the two, so much so that Scrapy's parsel library parses CSS selectors and transforms then into the equivalent XPath for execution
To the best of my knowledge, CSS selectors care only about the structure of the page, lightly dipping its toes in the content only for attributes and things like :first-char and :first-line
Re:Web scraping via JavaScript runtime heap snapshots
by chrismeller on Friday April 29, 2022 @05:04PM
(#31210579)
Well that is 100% originally an XPath selector (:nth-child), so kudos if CSS selectors support it now.
Still, using // instead of multiple *’s (and the two divs) still seems better for longer-term scraping.
We all know the display of the page and the structure of the page should be mutually exclusive, so why would you base your selectors on display? Particularly if you’re looking for something on a semantically designed page, why would I look for an .article, a class that may disappear with the next redesign, when they’re unlikely to stop using the article HTML tag?