Kaitai: Describe the structure of data, not how you read or write it

Re:Kaitai: Describe the structure of data, not how you read ... by convolvatron on Saturday December 05, 2020 @03:57PM (#25318274)

I've looked at this problem quite a bit over the years...I agree with you completely. there isn't anything fundamental here, just the normal cultural adoption issues, usability, etc. There may be some compilation/complexity issues around formats with variable length fields and self-description, but certainly less problematic than general purpose programming.

I really wish though that there were more traction here, as I really believe that we should be quite prepared do deal with bits and not protobufs by default. nothing wrong with protobufs for quite a number of uses. I just don't know why people are so afraid of and/or biased against bit strings.

[ Parent | on HN ]

Re:Kaitai: Describe the structure of data, not how you read ... by joshspankit on Saturday December 05, 2020 @04:33PM (#25318606)

Through this thread I really feel like I’m missing something.

Are we not talking about writing binary files that conform to the spec?

Like, in the case of a GIF: simply writing valid garbage data should produce a file that presents as a valid GIF with noise for the image. Similarly: reading the file through the parser and writing it out unmodified should create an identical file (assuming no stenography).

Right?

[ Parent | on HN ]

Re:Kaitai: Describe the structure of data, not how you read ... by IceDane on Saturday December 05, 2020 @07:10PM (#25319678)

You're right. The person saying this would be a breakthrough doesn't understand what this is doing.

There are already similarly declarative tools which can accomplish this. Haskell has binary parsing libraries which work similarly and give you both reading and writing capabilities.

[ Parent | on HN ]

Re:Kaitai: Describe the structure of data, not how you read ... by spullara on Saturday December 05, 2020 @05:21PM (#25318970)

I honestly don't get it either. The inverse of the read spec is the write spec. My guess having not dug deeply is that they don't distinguish between required and optional fields, that said, they should still be able to write what they have based on the read spec but could potentially be still an invalid file.

[ Parent | on HN ]

Re:Kaitai: Describe the structure of data, not how you read ... by armsforsharks on Saturday December 05, 2020 @06:53PM (#25319574)

It's a really interesting idea! I'm also surprised there hasn't been much traction here. I've started a Rust library for this: https://github.com/sharksforarms/deku

It's a declarative bit-level symmetrical reader writer generator library.

[ Parent | on HN ]