News for hackers. Stuff that matters.
Re:Compiler Construction by Niklaus Wirth [pdf] by dbcurtis on Friday January 31, 2020 @04:50PM (#22206200)
The problem with LALR parser generators is that you end up with an LALR parser. Which is fine as long as the input is error-free, but trying to get a reasonable error message out of any LALR parser is about as much fun as repeatedly poking yourself in the eye with a sharp stick.

FLEX, on the other hand, is perfect for the job. I can't imagine why anyone would hand-write a lexer.

Re:Compiler Construction by Niklaus Wirth [pdf] by jcranmer on Friday January 31, 2020 @05:54PM (#22206789)
> FLEX, on the other hand, is perfect for the job. I can't imagine why anyone would hand-write a lexer.

Flex introduces a new dependency to your project, which involves generating source files at build time. This is a lot of complexity in your build system if this is the first time you need it. And it also by default uses a style that isn't really modern--it's more annoying to use if you want to use it in a multithreaded programming, or if you want to have multiple lexers in one program. (Or you're not even using C/C++ in the first place!).

Furthermore, lexers are actually really easy to write. Most tokens [1] are recognizable within a character or two, and usually amount to a regex on the order of [a-z][a-z0-9_]+ or "([^\"]|\.)*" -- the sorts of regexes that have very trivial implementations to handcode. The lexer I wrote for a JSON-ish file format is 128 lines of code. Sure, it'd be more like 30 lines of code if I used an automatic generator, but running into any issues integrating the automatic generator into my workflow would toss out the benefits of saving 100 lines of code.

[1] Excluding keywords, but keywords are usually implemented by parsing it as an identifier and looking for the identifier in a hashtable of known keywords.

Re:Compiler Construction by Niklaus Wirth [pdf] by beagle3 on Saturday February 01, 2020 @12:38PM (#22211786)
In my experience, re2c is an improvement on lex/flex in most ways, but I agree - flex is indeed nicer than hand-rolled in every way (except if you are Arthur Whitney - his hand rolled lexers are a marvel)