Re:Data Structures in the Fleet Editor
by dataangel on Tuesday February 22, 2022 @11:00AM
(#30429502)
Interesting trick, but I assume this only works well if you're smart about what you put in the parser state. If line number is in there then it will never converge with old snapshots.
[ Parent
| on HN ]
The parser state for Rouge for all the parsers provided out of the box consists of one or more symbols representing the current state in a state machine, so most of the time it's a single symbol. For some complex parsers it'll be an array of a few symbols (e.g. if the parser delegates to another - an example would be that I layer parsers so that comments get parsed by a Markdown parser, and that again can have block comments specifying another language)
Worst case scenario is if you open something like a block comment which doesn't close and doesn't hit an error condition which resets the parser state. If you need to consider large files that's a reason to re-parse the buffer lazily in the background...
But, yes, if you write your own parsers you certainly would also need to ensure the state is as compact as possible.