Interesting benchmarks. One thing I'd love to see separated out: how much of the gap is syntax vs. how much is the model just having seen far less of the "fixed" form in training. A simpler grammar with zero corpus behind it could easily perform worse in practice.