News for hackers. Stuff that matters.
Re:Tree Sitter and the Complications of Parsing Languages by matklad on Wednesday November 24, 2021 @10:52AM (#29331194)
The thing is, for IDE purposes “production ready” has a different definition. The thing shouldn’t have 100% parity with the compiler, it should be close enough to be useful, and it must be resilient. This is definitely not trivial, but is totally within the reach of a single person.

> And this doesn't even count error recovery / dealing with broken code ...

With a hand written parser, you mostly get error resilience for free. In rust-analyzer’s parser, there’s very little code which explicitly deals with recovery. The trick, is, during recursive descent, to just not bail on the first error.

Re:Tree Sitter and the Complications of Parsing Languages by jhck on Wednesday November 24, 2021 @11:28AM (#29331698)
Those are some very nice insights, thanks for sharing them! Can you recommend a good resource on writing a parser by hand that doesn't bail on the first error? Or would you instead suggest studying the source code for e.g. the rust-analyzer parser?
Re:Tree Sitter and the Complications of Parsing Languages by dunham on Wednesday November 24, 2021 @12:30PM (#29332570)
I can't answer this well and don't know of any resources, but I have seen it before in the parser for sixten:

    https://github.com/ollef/sixten/blob/60d46eee20abd62599badea85774a9365c81af45/src/Frontend/Parse.hs#L458
In that case, they're parsing a haskell-like language and can use indentation as a guide for how far to skip ahead.

In a C-like language, I'd imagine you'd use braces or semicolons to see how far to skip ahead - the error bubbles up to a parser that knows how to recover, like say a statement or function body, it scans ahead to where it thinks its node ends and returns an error node, allowing the parent to continue.

Re:Tree Sitter and the Complications of Parsing Languages by matklad on Wednesday November 24, 2021 @01:05PM (#29333026)
I don’t have a good resource, but https://m.youtube.com/watch?v=0HlrqwLjCxA is probably an OK start.
Re:Tree Sitter and the Complications of Parsing Languages by jhck on Wednesday November 24, 2021 @02:12PM (#29333891)
Thanks a lot, I'll give it a watch!

Also, thanks dunham for the sixten suggestion!