News for hackers. Stuff that matters.
Re:Parsley: a simple language for extracting structured data... by fizx on Sunday January 11, 2015 @11:27PM (#8872509)
Hi everyone. Library author here. I worked on this a bunch back in maybe 2009-2010. It's inspired by the work I did with tectonic on selectorgadget.

So my current thinking on the idea is reflected in https://github.com/fizx/pquery. PQuery addresses some weaknesses of Parsley by embedding the ideas in Javascript.

(1) Parsley isn't turing-complete, and many web pages are ugly, so you often have to resort to pre/post-processing in some scripting language. I never was able to get sufficient power out of a purely declarative language.

(2) Javascript environments are readily available (even in embedded form), and are more accessible than C.

(3) If your crawler already executes Javascript to render dynamic pages, then running more Javascript in that environment is pretty easy.

I guess I'm a little late to the thread (yay weekends) but I'll answer any questions people may have.

Re:Parsley: a simple language for extracting structured data... by tectonic on Sunday January 11, 2015 @11:39PM (#8872529)
Those were fun times :)

Ruby bindings: https://github.com/fizx/parsley-ruby

Python bindings: https://github.com/fizx/pyparsley