News for hackers. Stuff that matters.
Re:Pro scraping with Node.JS by simonw on Sunday January 23, 2011 @02:46AM (#2132010)
I was intrigued to see what CSS selector engine it was using...

https://github.com/chriso/node.io uses https://github.com/harryf/node-soupselect

https://github.com/harryf/node-soupselect is a port of my https://github.com/simonw/soupselect library for Python

https://github.com/simonw/soupselect is a port of my getElementsBySelector function for JavaScript: http://simonwillison.net/2003/Mar/25/getElementsBySelector/

I'm always surprised to see that code still being used - it's the least complete selector library out there by a long way.

Re:Pro scraping with Node.JS by harryf on Sunday January 23, 2011 @05:13AM (#2132172)
> I'm always surprised to see that code still being used - it's the least complete selector library out there by a long way.

For this use case being complete doesn't matter so much as users are just after a handy way pull content out of a page. And having "got intimate" with the source while porting it, it's a really elegant piece of code - have the impression that try to make it do more would ruin it.

Re:Pro scraping with Node.JS by chrisohara on Sunday January 23, 2011 @02:57AM (#2132018)
Hi Simon, great libs - thanks! there's been many improvements added to node-soupselect and node.io though - the API is here: https://github.com/chriso/node.io/wiki/API---CSS-Selectors-a...
Re:Pro scraping with Node.JS by simonw on Sunday January 23, 2011 @03:11AM (#2132033)
Oh nice - the .rawtext and .striptags methods are particularly useful.