Re:Compiler Construction: The Art of Niklaus Wirth (2000) [pdf]
by chubot on Wednesday March 21, 2018 @06:40PM
(#16643090)
Interesting, thanks for the feedback. It seems like the lesson is analogous to what I learned from Crockford's JavaScript parser.
That is, regardless of whether you're doing it top-down or bottom-up, using an operator precedence algorithm for statements is painful.
I think the reason is simply that there is a fundamental relationship between the language you are parsing (the data) the algorithm you choose to parse it with (the code). Those 2 things are tightly linked.
I think this isn't emphasized enough in introductory texts. They usually only have time to teach one parsing method, and the language being parsed is chosen to fit that. Or maybe you choose one problem like shift-reduce conflicts, but you don't get the comparison / contrast to other methods.
I've done a full Pascal-like language using bottom up precedence parsing in the past. It was my first compiler project about 25 years ago or so, and "invented" precedence parsing with the help of a symbolic expression evaluator in Prolog in a book I picked up that I tried to figure out how to translate to Pascal.
It was very disappointing to find out it was a well known algorithm, but it was very interesting to try to expand it to handle a full language, which involved funky things like handling operators with different structure than just simple single-operand prefix/postfix + infix operators. E.g. I implemented a dual-arity prefix operator to handle for/while/if/ternary for example. So "while" took two expressions: the condition and a block.
The hard part was not handling the control structures, but getting the priorities right.
I'm not sure I'd do that again. My Ruby parser uses recursive descent for the top level productions.
But it was fascinating to do, as it did make for a very compact parser.
> It seems to me that the statement language and expression language are separate things,
I think if you go down that route, then you are implicitly deciding that everything is an expression. In e.g. Ruby that is true, though many constructs just returns nil (e.g "somevar = while condition; end" is valid). But given that many of the "operators" are only allowed at specific levels, it makes less sense to use a precedence parser anyway (e.g. "class" can't occur in a method body for example).
For me, those two are the biggest considerations to whether or not I'd consider a precedence parser: Whether or not everything is an expression, and which keywords are only allowed in specific contexts.
But even if it's feasible without too many contortions, it may still be very painful.