News for hackers. Stuff that matters.
Re:Parsley: a simple language for extracting structured data... by fizx on Sunday January 11, 2015 @11:27PM (#8872509)
Hi everyone. Library author here. I worked on this a bunch back in maybe 2009-2010. It's inspired by the work I did with tectonic on selectorgadget.

So my current thinking on the idea is reflected in https://github.com/fizx/pquery. PQuery addresses some weaknesses of Parsley by embedding the ideas in Javascript.

(1) Parsley isn't turing-complete, and many web pages are ugly, so you often have to resort to pre/post-processing in some scripting language. I never was able to get sufficient power out of a purely declarative language.

(2) Javascript environments are readily available (even in embedded form), and are more accessible than C.

(3) If your crawler already executes Javascript to render dynamic pages, then running more Javascript in that environment is pretty easy.

I guess I'm a little late to the thread (yay weekends) but I'll answer any questions people may have.

Re:Parsley: a simple language for extracting structured data... by tectonic on Sunday January 11, 2015 @11:39PM (#8872529)
Those were fun times :)

Ruby bindings: https://github.com/fizx/parsley-ruby

Python bindings: https://github.com/fizx/pyparsley

Re:Parsley: a simple language for extracting structured data... by tbatchelli on Sunday January 11, 2015 @08:32PM (#8872093)
Not to be confused by Christopher Grand's clojure parser library:

https://github.com/cgrand/parsley

Re:Parsley: a simple language for extracting structured data... by nvader on Sunday January 11, 2015 @11:10PM (#8872463)
(also replying to siblings)

Parsley is just so good a name. It has the word parse as a kangaroo, and it evokes the image of fresh, green, edible.

I just ran

    grep /usr/share/dict/words -e "^pars[^']*[^s]$"
to find the following list of words beginning with pars

    parse
    parsec
    parsed
    parser
    parsimony
    parsing
    parsley
    parsnip
    parson
    parsonage
I think I might call my next parsing-oriented tool either parson or parsnip. :)
Re:Parsley: a simple language for extracting structured data... by bshimmin on Monday January 12, 2015 @01:56AM (#8872786)
I was interested in "kangaroo"; Wikipedia suggests a kangaroo word should, in addition to having the same letters and in the same order as its parent, also have the same meaning (eg. masculine -> male), so "parse" isn't quite a kangaroo for "parsley", apparently. It'd be a moderately fun little challenge to write a little program to find some kangaroos - though Wiktionary (predictably) has a nice list already here: http://en.wiktionary.org/wiki/Appendix:Kangaroo_words

"Parsley" is indeed a great name. I'll contribute another Parsley - a Flex framework of yore also bore that name.

Re:Parsley: a simple language for extracting structured data... by nvader on Monday January 12, 2015 @05:32AM (#8873163)
The reason I thought it was a kangaroo was specifically because it was being used as a name for a parsing library.

After a library is named "Parsley", the list of meanings for Parsley now includes "A parsing library", and so I see it as a kangaroo for Parse.

Re:Parsley: a simple language for extracting structured data... by JadeNB on Monday January 12, 2015 @02:34PM (#8875903)
> grep /usr/share/dict/words -e "^pars[^']<star>[^s]$"

(I changed a literal asterisk to <star> to avoid formatting.) You get a few more options if you remove the anchor at the beginning:

    grep /usr/share/dict/words -e "pars[^']<star>[^s]$"
For example, 'sparse' seems like a good name for a lightweight parser library.
Re:Parsley: a simple language for extracting structured data... by abecedarius on Monday January 12, 2015 @11:57AM (#8874907)
I'm afraid there's https://github.com/darius/parson already too. (I went through the same process to name it -- there's not much left in that well, is there?)
Re:Parsley: a simple language for extracting structured data... by nvader on Sunday January 11, 2015 @11:12PM (#8872470)
Oops, looks like I was beaten by cgrand himself. https://github.com/cgrand/parsnip
Re:Parsley: a simple language for extracting structured data... by allr on Sunday January 11, 2015 @08:33PM (#8872098)
Re:Parsley: a simple language for extracting structured data... by jshprentz on Sunday January 11, 2015 @09:19PM (#8872196)
Or Parsley: a pattern-matching language based on OMeta and Python

https://github.com/python-parsley/parsley

Re:Parsley: a simple language for extracting structured data... by zo1 on Monday January 12, 2015 @05:27AM (#8873150)
Or a PEG grammar parser written in python:

https://pypi.python.org/pypi/Parsley

Re:Parsley: a simple language for extracting structured data... by fizx on Monday January 12, 2015 @04:43PM (#8876605)
First commit on fizx/parsley was in 2008. The library was open-sourced in 2009. This predates all of the name conflicts so far mentioned, though no doubt someone used the name before in some capacity.
Re:Parsley: a simple language for extracting structured data... by xchaotic on Monday January 12, 2015 @03:40AM (#8872949)
Why not start with Xpath and CSS selectors and pre-/post-process in js as needed?
Re:Parsley: a simple language for extracting structured data... by divideby0 on Sunday January 11, 2015 @08:32PM (#8872090)
Looks pretty awesome, esp the clean DSL for your page model, but it seems like most of the documentation might be missing. How sophisticated is the crawler portion? Does it support Nutch-style generators that crawl more frequently updated pages more frequently? Or is it more designed for focused, one-off crawls a la Scrapy?
Re:Parsley: a simple language for extracting structured data... by fizx on Sunday January 11, 2015 @11:17PM (#8872485)
The crawler portion is about as sophisticated as `wget -R`.
Re:Parsley: a simple language for extracting structured data... by keyle on Sunday January 11, 2015 @09:27PM (#8872212)
Shameless plug... https://github.com/keyle/json-anything did this a while back
Re:Parsley: a simple language for extracting structured data... by zo1 on Monday January 12, 2015 @05:38AM (#8873177)
Looks pretty neat, +1 for concept & implementation. When I get time I'll be trying it out.

I'd also like to give some sort of -1 for the recycled library name, though it's not a technical nit pick, just a personal one. The name of the library is mostly dominating the discussion here at the moment, and that's a shame.

Re:Parsley: a simple language for extracting structured data... by grogenaut on Sunday January 11, 2015 @11:45PM (#8872546)
Interesting straight duh idea (as in duh why didn't I think of that). Would use it. Wonder how it handles looping.

Note 3/4 of the links on the main page are to not yet created wiki pages. Looking forward to it, or just writing it for myself in go :)

Re:Parsley: a simple language for extracting structured data... by keyle on Sunday January 11, 2015 @09:49PM (#8872277)
Btw I had this idea of a web query language but never went anywhere https://gist.github.com/keyle/10951106 Have a look and let me know your thoughts if any.
Re:Parsley: a simple language for extracting structured data... by dominotw on Sunday January 11, 2015 @10:13PM (#8872338)
Looks a lot like using xml linq provider to query html page.
Re:Parsley: a simple language for extracting structured data... by SigmundA on Monday January 12, 2015 @11:54AM (#8874893)
Using https://github.com/MindTouch/SGMLReader you can use Linq to XML or anything else that accepts an XmlReader interface in .Net.
Re:Parsley: a simple language for extracting structured data... by mkoryak on Sunday January 11, 2015 @08:57PM (#8872151)
last commit in 2013. Is it a done, or is it just no longer maintained?
Re:Parsley: a simple language for extracting structured data... by fizx on Sunday January 11, 2015 @11:16PM (#8872482)
I don't know. Perhaps a little of each. See my other comments here.