The Construct library for Python can do it:

https://construct.readthedocs.io/en/latest/intro.html#exampl...

I’ve long searched for something better than Construct, but so far I have yet to find even an equal.

It can, but it can get incredibly slow for large formats. I was using it to reverse engineer some binary game formats but the parser would take a couple of minutes to complete. I rewrote it using struct and that time dropped to a few seconds. Useful for probing an unknown format, but I prefer the 010 editor since it’s more interactive.

Ooh, exciting! I built a parser [1] for AIS messages [2], a quirky ship-to-ship protocol. My lower-level stuff always felt clumsy to me. I'll have to see if this cleans it up.

[1] https://github.com/wpietri/simpleais [2] https://gpsd.gitlab.io/gpsd/AIVDM.html

BinData (https://github.com/dmendel/bindata) is a Ruby gem for this, basically using a DSL in Ruby to declaratively define binary data formats that can be both read and written.

I wrote a sort-of adjacent library for Go at one point. I’m a bit stuck trying to figure out exactly what to do on 2.0 but it has a lot of Kaitai like features including an expression language for transforming things (on master version) and it supports writing structures out.

https://github.com/go-restruct/restruct

I have been working on Deku: a declarative binary reading and writing: bit-level, symmetric, serialization/deserialization library. https://github.com/sharksforarms/deku

and still there is no generator that creates efficient and partialy streamed readers/writers for high performance protocols or resource constraint environments (as less pre allocations as possible, zero copy concepts,..., streamed reading/writing, good inline possibility, ...)

100% fix formats with no self reference or for example checksums that sits in front of the checksumd data (no streaming possible)...

Recordflux?