News for hackers. Stuff that matters.
Re:How to Parse Ruby by peterwoo on Thursday February 07, 2013 @02:30PM (#5184022)
Python is not context free.
Re:How to Parse Ruby by Scaevolus on Thursday February 07, 2013 @03:11PM (#5184240)
Indentation is handled by the lexer, not the parser.

The parser is context free-- LL(1), actually.

The lexer maintains one extra stack to track indentations, meaning it's more of a PDA than a DFA, but the actual complication is mild.

Re:How to Parse Ruby by peterwoo on Thursday February 07, 2013 @05:43PM (#5185008)
The language is not context free, although I agree there is a parsing solution which is not terribly complex.
Re:How to Parse Ruby by lucian1900 on Friday February 08, 2013 @04:53AM (#5187084)
It is context free. INDENT and OUTDENT are directly equivalent to { and } in other grammars.
Re:How to Parse Ruby by peterwoo on Saturday February 09, 2013 @02:01AM (#5191965)
The difference is that INDENT and OUTDENT have to be defined by context-sensitive productions. Even if this occurs in what would otherwise be the "lexing" stage of the parser.
Re:How to Parse Ruby by masklinn on Thursday February 07, 2013 @02:51PM (#5184135)
And there's no way java is. Not sure about Go.
Re:How to Parse Ruby by lucian1900 on Thursday February 07, 2013 @04:48PM (#5184724)
The Java language specification includes a description of the language as a context free grammar.

That alone does not make it a nice language, of course.

Re:How to Parse Ruby by masklinn on Friday February 08, 2013 @02:08AM (#5186689)
> The Java language specification includes a description of the language as a context free grammar.

I can only assume that it's lying its pants off, as I noted in an other subthread `(a) - (b)` can't be parsed (to an AST) without knowing the identity of `a` in the current scope: if it's a type the expression is a cast of `-(b)` to `a`, if it's a value it's a subtraction of `(b)` from `(a)`.

Re:How to Parse Ruby by peterwoo on Friday February 08, 2013 @03:46AM (#5186931)
It can be parsed to an AST -- as you note, it can be parsed into two ASTs. It's called an ambiguous grammar, which can still be context free.
Re:How to Parse Ruby by masklinn on Friday February 08, 2013 @05:03AM (#5187104)
> It can be parsed to an AST -- as you note, it can be parsed into two ASTs.

By "it can't be parsed to an AST" I meant "it can't be parsed to a useful AST used to run or generate code" on its own, it needs to be disambiguated through contextual information before anything can be done with the prospective subtree. Sorry for the lack of clarity.

> It's called an ambiguous grammar, which can still be context free.

The only way to disambiguate is to provide expression context (namely visible name bindings at this point), it's essentially the same problem Eli Bendersky described when talking about C grammar's context-sensitivity: http://eli.thegreenplace.net/2007/11/24/the-context-sensitiv...