Re:Parsley: a simple language for extracting structured data...
by tectonic on Sunday January 11, 2015 @11:39PM
(#8872529)
Those were fun times :)
Ruby bindings: https://github.com/fizx/parsley-ruby
Python bindings: https://github.com/fizx/pyparsley
So my current thinking on the idea is reflected in https://github.com/fizx/pquery. PQuery addresses some weaknesses of Parsley by embedding the ideas in Javascript.
(1) Parsley isn't turing-complete, and many web pages are ugly, so you often have to resort to pre/post-processing in some scripting language. I never was able to get sufficient power out of a purely declarative language.
(2) Javascript environments are readily available (even in embedded form), and are more accessible than C.
(3) If your crawler already executes Javascript to render dynamic pages, then running more Javascript in that environment is pretty easy.
I guess I'm a little late to the thread (yay weekends) but I'll answer any questions people may have.