News for hackers. Stuff that matters.
Re:Writing parsers like it is 2017 by ioquatix on Tuesday August 15, 2017 @07:01AM (#15017057)
I'm surprised no one has mentioned Ragel: http://www.colm.net/open-source/ragel/

Ragel generates very fast code, is modular by design and has good error handling. It supports both compiled and scripted languages (e.g. can generate both Ruby and C) which is useful if you need a fallback.

I've really enjoyed using it to implement the following:

- A template language for Ruby: https://github.com/ioquatix/trenni/blob/master/parsers/trenn...

- A SGML parser for Ruby: https://github.com/ioquatix/trenni/blob/master/parsers/trenn...

- A HTTP V1 protocol parser: https://github.com/kurocha/async-http/blob/master/source/Asy...

- A URI parser: https://github.com/kurocha/uri/blob/master/source/URI/RFC398...

I'll admit, it does take a while to understand how to correctly handle ambiguity when dealing with callbacks/events during parsing, but generally speaking, once you get a bit of experience with how things work, it becomes a very powerful tool in your toolbox.

Re:Writing parsers like it is 2017 by vram22 on Tuesday August 15, 2017 @08:12AM (#15017447)
IIRC, Zed Shaw used Ragel to create a parser for Mongrel or Mongrel2 (the web servers).

Update: Checked it, he did:

https://en.wikipedia.org/wiki/Mongrel_(web_server)

https://en.wikipedia.org/wiki/Mongrel2

Re:Writing parsers like it is 2017 by allengeorge on Tuesday August 15, 2017 @07:10AM (#15017090)
I thought that Ragel was no longer open-source, and supported C/C++ only (admittedly, the second point isn't that big a deal, given the article and its audience).
Re:Writing parsers like it is 2017 by justincormack on Tuesday August 15, 2017 @07:19AM (#15017153)
No. It changed from GPL to MIT license, but did not go closed source. http://www.colm.net/open-source/ragel/
Re:Writing parsers like it is 2017 by jwilk on Tuesday August 15, 2017 @07:21AM (#15017159)
Cloudbleed was caused by a bug in a Ragel-based parser.
Re:Writing parsers like it is 2017 by rhencke on Tuesday August 15, 2017 @08:13AM (#15017457)
It was not caused by a bug in Ragel, though.
Re:Writing parsers like it is 2017 by staticassertion on Tuesday August 15, 2017 @11:28AM (#15019121)
This is always going to lead to the same place - whether it's the developer's fault or the tool's.

The author of this post wants uncompromising safety, ragel does not allow for that - programmer error can introduce memory unsafety. They even explicitly call out Cloudbleed.

Re:Writing parsers like it is 2017 by wuch on Tuesday August 15, 2017 @08:52AM (#15017693)
To provide a little bit more context, here is detailed description of this bug [0]. Curiously, they suggest that problem could be avoided by changing the check for EOF to ">=". This is not true at all in case of C, at least as far as language semantics is concerned. When pointer goes two past the end of array you are already in undefined behaviour land.

[0] https://blog.cloudflare.com/incident-report-on-memory-leak-c...

Re:Writing parsers like it is 2017 by ioquatix on Wednesday August 16, 2017 @03:05AM (#15025498)
I think it's valid in C/C++ to go 1 past the end of an array: https://stackoverflow.com/questions/988158/take-the-address-...
Re:Writing parsers like it is 2017 by striking on Tuesday August 15, 2017 @01:15PM (#15020190)
More specifically:

> The Ragel code we wrote contained a bug that caused the pointer to jump over the end of the buffer and past the ability of an equality check to spot the buffer overrun.

Re:Writing parsers like it is 2017 by ioquatix on Wednesday August 16, 2017 @03:07AM (#15025506)
A simple `assert(p<pe);` in a debug build would have caught the issue. I think you could argue that ragel should include something like this by default.