Rewriting the Ruby Parser

Re:Rewriting the Ruby Parser by e4m2 on Tuesday June 13, 2023 @11:00AM (#36311245)

Interestingly, the actual syntax tree and related structures/functions are generated from the config.yml file and the templates inside the bin directory. They are using a custom template language written in Ruby, here's an example of how they do enum stringification: https://github.com/ruby/yarp/blob/main/bin/templates/src/tok.... Obviously this isn't a novel idea, but IMO this kind of design goes a long way to support their maintainability argument, especially in C.

Also,

> CRuby actually ships with 90 encodings (as of 3.3)

This is asinine.

[ Parent | on HN ]

Re:Rewriting the Ruby Parser by kddeisz on Tuesday June 13, 2023 @11:50AM (#36312054)

The templating is definitely an effort to keep the maintainability in check. It's also because we're planning on keeping a grammar around to generate test cases for us with fuzzing/other algorithms.

The encodings is a bit historical, to my knowledge Ruby is the only major language that was developed by someone whose language did not work with ASCII. So the first encodings written in Ruby were the old Windows pages.

[ Parent | on HN ]

Re:Rewriting the Ruby Parser by cremno on Tuesday June 13, 2023 @12:32PM (#36312741)

>> CRuby actually ships with 90 encodings (as of 3.3)

>This is asinine.

Overall there are even more: 103. But then again 1.9.2 only has 85 and 95 overall. Also one of those new 'overall' ones is the EBCDIC code page for US/Canada.

[ Parent | on HN ]

Re:Rewriting the Ruby Parser by munificent on Tuesday June 13, 2023 @01:38PM (#36313747)

> Interestingly, the actual syntax tree and related structures/functions are generated from the config.yml file and the templates inside the bin directory.

That makes a lot of sense. ASTs tend to be a very dumb but fairly verbose data structure. And, in C in particular where everything is more verbose, there ends up being a ton of boilerplate in the AST nodes.

In a language like Ruby with a very rich syntax, you end up needed a ton of different AST nodes.

Generating those from a simple declarative format makes maintaining that much easier.

[ Parent | on HN ]

Re:Rewriting the Ruby Parser by vidarh on Tuesday June 13, 2023 @02:02PM (#36314097)

> a custom template language written in Ruby

This makes it sound like it's some template language specific to this project, but ERB is the dominant templating language for Ruby because an ERB implementation is in the standard library.

[ Parent | on HN ]