Re:Tree-sitter: an incremental parsing system for programmin...
by dcreager on Monday February 22, 2021 @12:49PM
(#26227615)
[EDITED to make the example actually line up with OP's test]
There's no symbol table in the parser, so at parse time, we don't distinguish those cases:
$ cat test.rb
module Test
def test1
x = 14; x
end
def test2
y = 14; x
end
end
$ tree-sitter parse test.rb
(program [0, 0] - [9, 0]
(module [0, 0] - [8, 3]
name: (constant [0, 7] - [0, 11])
(method [1, 2] - [3, 5]
name: (identifier [1, 6] - [1, 11])
(assignment [2, 4] - [2, 10]
left: (identifier [2, 4] - [2, 5])
right: (integer [2, 8] - [2, 10]))
(identifier [2, 12] - [2, 13]))
(method [5, 2] - [7, 5]
name: (identifier [5, 6] - [5, 11])
(assignment [6, 4] - [6, 10]
left: (identifier [6, 4] - [6, 5])
right: (integer [6, 8] - [6, 10]))
(identifier [6, 12] - [6, 13]))))
In both cases the bit after the semicolon just parses as (identifier).For some use cases (e.g. syntax highlighting, depending on your colorization rules) it doesn't matter, and so we don't want to pay the cost. If it does matter (like in an actual implementation), then you'd have to implement this yourself and drive it by the parse tree you get from tree-sitter.
Re:Tree-sitter: an incremental parsing system for programmin...
by chrisseaton on Monday February 22, 2021 @12:50PM
(#26227633)
Right you could just have a phase to fix-it-up after parsing. Much better than trying to shoe-horn an imperative action into a nice more-pure parser. Great idea!
Re:Tree-sitter: an incremental parsing system for programmin...
by tclem on Monday February 22, 2021 @11:20PM
(#26233778)
So what's cool is that while we don't handle that during parsing, you can use another set of tree-sitter features to do tree queries to achieve this. Here's the query for detecting Ruby locals: https://github.com/tree-sitter/tree-sitter-ruby/blob/32cd5a0... and here's some better documentation for how the query language works: https://tree-sitter.github.io/tree-sitter/syntax-highlightin....
I really want to try tree-sitter for using in an actual Ruby implementation because it's so beautiful!