Why not use PEGs? In short, they don't actually remove ambiguity from your grammar, but rather hide it in ways that are difficult to reason about. You're still dependent on code as a definition of the language that you accept, and worse, that code is very fragile in the face of refactoring. Further, the lexical analysis step is generally the source of most ambiguity, and most parser combinator libraries make it extremely inconvenient to parse something that isn't a character stream. (Hammer is actually worse than most here; it's not possible to use it to parse anything other than streams of bytes, though perhaps it can be excused because of the limitations of C)
I've written a tool that converts a declarative specification of a PEG grammar into a parser, although I haven't released it yet. It works great, I've used it on a number of real-world structured data formats.
Edit: cut out the first paragraph after checking the user name. Hi TQ.
In that sense, LALR parser generators (e.g. yacc) are also never ambiguous, because even if your grammar is ambiguous in the formal sense, yacc will produce a working parser with well-defined resolution of conflicts...
But the reality is,
E = E `+` E
is intrinsically ambiguous, no matter how you want to spin it... the only difference is, that LALR generators point that out, whereas PEG generators sweep it under the rug.To lean on that means you have hidden information: your grammar is BNF + LALR, or BNF + GLL, not just BNF. PEGs, by contrast, are always PEGs. What you see is what you get.
A grammar in PEG format will always give you one parse, and which parse is predictable. If you want a different parse, you have to rewrite it. Indeed, as I'm sure you know, your example grammar isn't valid in the original PEG formalism, which prohibits immediate left recursion. Automatic rewrites into an intermediate rule are the leading method of allowing it.
Ambiguity is a well-defined concept in grammars, and PEGs aren't.
I've noticed that a lot of CFG enthusiasts don't like this about PEGs. They consider it inelegant, unprincipled. Some of that is aesthetic, some is unfamiliarity, and some is sunk cost: none of it actually engages with the formal expressive power of PEGs, nor their ergonomics as a practical tool for development.
I, on the other hand, am primarily interested in using grammars to parse programming languages, which are essentially a human form of communication (computers could use bitcode or lisp-style ASTs, no need for syntax).
I'm not an expert on PEGs, so this is copy-pasted from Wikipedia page on PEGs, I hope it's valid - the classical dangling else ambiguity.
S ← 'if' C 'then' S 'else' S / 'if' C 'then' S
Now, there is an obvious ambiguity here, for a human reader (mediated, of course, by tabs). If you're using a formalism that wishes that ambiguity away, it just means that the formalism is a non-ideal one. What I like about LALR parser generators is, that they will explicitly alert you of this kind of human-level ambiguities.It's like saying there's an ambiguity in "not a and b". Sure, if you don't know the precedence assigned to 'not' and 'and' in your language. But you're supposed to learn those things.
Your not knowing how PEGs work is a weak argument against using them.
For more info on the issues with PEGs -- and a paper showing how to correctly translate general LL(1p) grammars to PEG -- see:
https://jeffreykegler.github.io/Ocean-of-Awareness-blog/indi...
-- returns DER object pattern that captures inner value
local function Cobject(identifier, patt)
local match
if lpeg.type(patt) then
match = function (s)
return lpeg.match(patt * -P(1), s)
end
elseif type(patt) == "function" then
match = patt
elseif patt == nil then
match = function (s)
return s
end
else
error(sformat("expected function, pattern or nil, got %s", type(patt)), 2)
end
return Cmt(identifier, function (s, pos)
local n, pos = assert(unpacklength(s, pos))
local s1 = s:sub(pos, pos + n - 1)
pos = pos + n
return (function (pos, v, ...)
if v then
return pos, v, ...
else
return false
end
end)(pos, match(s1))
end)
end
local BIT_STRING = Cobject(P"\x03", function (s)
local pad = s:byte(1) -- first octet is number of padding bits
assert(pad == 0, "BIT STRING not octet aligned") -- we only support DER
return s:sub(2)
end)
local IA5String = Cobject(P"\x16")
local OID = function (oid)
if oid then
local s = packoid(pkix.txt2oid(oid))
return P(sformat("\x06%s%s", packlength(#s), s)) * Cc(oid)
else
return Cobject(P"\x06", function (s)
return assert(unpackoid(s))
end)
end
end
local SEQUENCE = function (patt)
return Cobject(P"\x30", patt)
end
local TBSCertificate = SEQUENCE(Ct(
Cg(Version, "version") *
Cg(CertificateSerialNumber, "serialNumber") *
Cg(AlgorithmIdentifier, "signature") *
Cg(Name, "issuer") *
Cg(Validity, "validity") *
Cg(Name, "subject") *
Cg(SubjectPublicKeyInfo, "subjectPublicKeyInfo") *
Cg(UniqueIdentifier(1), "issuerUniqueID")^-1 *
Cg(UniqueIdentifier(2), "subjectUniqueID")^-1 *
Cg(Extensions, "extensions")^-1 *
Cg(P(1)^1, "trash")^-1
))
local Signature = BIT_STRING
local Certificate = SEQUENCE(Ct(
Cg(TBSCertificate, "tbsCertificate") *
Cg(AlgorithmIdentifier, "signatureAlgorithm") *
Cg(Signature, "signature")
))Yup.
Consider the much maligned (apache) httpd.conf and makefile. Their code was the source of truth for correctness. No matter horrible terrible their syntax, they would have been sufferable if they had grammars.
Bespoke parsing of expressions benefits from having an implicit grammar, like this OC. But threshold for needing explicit grammars is pretty low.
Prime examples are all the data exchange formats. Descriptions of JSON and CSV are short and naively reasonable, but insufficient. As we've seen, any ambiguity that can happen will happen.
This isn't bad advice across the board, but it is bad advice for many applications.
This is how you end up with context-sensitive formats, or ones where the semantics of a particular particle require running the parser, or worse. It exposes a rich surface area for exploitation with adversarial inputs.
That said, the advantages cited at the front of the article are all real: error recovery and decent error messages are unsolved problems with existing parser generators, although ANTLR is pretty good at it.
But, especially if you're developing the data format rather than just implementing it, I strongly suggest using something like Instaparse which can generate a parser directly from a grammar. Once this is solidified, it's reasonable to hand-roll something, but even there, I suggest using a nice parser combinator library, like hammer or Nom.
This provides more discipline: as long as you're coloring inside the lines, you'll end up with a parser which actually implements your spec. When you start adding functions to provide context, recover from errors, and provide meaningful feedback in the form of error messages for unexpected inputs, you'll know what you're doing: a classic recursive-descent parser can't provide the same conceptual separation between the parsing logic and the logic to support ergonomics or one-pass compiling or whatever add-ons you're putting in there.