News for hackers. Stuff that matters.
Re:Sylph: the programming language I want by gnuvince on Sunday March 01, 2015 @09:37AM (#9127148)
I like writing compilers, how do languages like Haskell and Python handle blocks-by-indentation? I figure you have some sort of INDENT token, but how do you know its width? Also, in specifying the grammar, with explicitly delimited blocks you just say `block := OPEN stmt_list CLOSE`. What about with indentation?
Re:Sylph: the programming language I want by TazeTSchnitzel on Sunday March 01, 2015 @09:55AM (#9127204)
I think they specify "INDENT foo DEDENT" and the lexer tracks indentation and issues INDENT/DEDENT as appropriate.
Re:Sylph: the programming language I want by falcolas on Sunday March 01, 2015 @10:57AM (#9127493)
This is correct. A bit of state (an indentation width stack) in the lexer creates the block boundaries as clearly for the interpreter as normal braces does.

Any problems in parsing are typically only encountered by stateless parsers.

Re:Sylph: the programming language I want by dochtman on Sunday March 01, 2015 @11:14AM (#9127585)
I have one (probably quite naive) implementation here:

https://github.com/djc/runa/blob/master/runac/parser.py#L54

(Runa uses tabs for indentation, not spaces, but it's basically the same thing.) Whitespace at the start of a line is a separate token, then I add a thin layer of processing that turns a change of indentation level (i.e. count chars in the token contents) into INDENT or DEDENT tokens.