Tree-sitter – a new parsing system for programming tools [video]

Re:Tree-sitter – a new parsing system for programming tools ... by mncharity on Tuesday October 16, 2018 @10:57PM (#18235781)

`externals` seem the (only) way to mix arbitrary code into the parsing, so...

Can `externals` call back into the parser? That is, is the parser reentrant?

Can `externals` be a zero-length assertions? That is, is there a progress requirement for success? And thus serve as arbitrary-computation parse-rejecting "semantic actions".

Do `externals` have access to GLR parse state?

[ Parent | on HN ]

Re:Tree-sitter – a new parsing system for programming tools ... by maxbrunsfeld on Tuesday October 16, 2018 @11:12PM (#18235839)

The Externals API is designed to only let you do things that are compatible with incremental parsing. It’s good for things like indentation based delimiters and automatic semicolon insertion.

You cannot call back into the parser; it’s just for tokenization.

You can produce zero length tokens, and we do use this feature a lot.

You dont have access to the parse state directly, but the external scanner is passed an array of Boolean values that indicates which external tokens are expected in the current state.

[ Parent | on HN ]