Re:On the Complexity and Performance of Parsing with Derivat...
by chubot on Sunday June 26, 2016 @11:23AM
(#11981145)
Agreed, although I would not say "most" can be formulated as CFGs. There is the lexer issue you mention, and that is mentioned in that article, but there are also tons of other issues in real languages.
Languages that are close to context free: Python, Java, JavaScript, R, Go, awk. (and I think Standard ML and OCaml)
Languages that are not even close: C, C++, Perl, Unix shell, make.
Languages I'm not sure about but which probably fall in the "no" category: PHP, Ruby, C#.
So I think that is enough to warrant some research into parsing tools for non-CFGs. (PEGs are interesting and I explored them for some of these applications)
That said, most programming languages can be formulated in an almost context-free way.
For real-world parsers, I think speed is just valued much higher than elegance, which often makes sense, especially for scripting languages where the parsing speed is directly proportional to program startup time (leaving caching aside).