News for hackers. Stuff that matters.
Re:Ask HN: When does it make sense to roll your own X? by CoolGuySteve on Sunday July 14, 2019 @11:03AM (#20433779)
For C structs, start with a uint32_t 4 char code identifier and a size field. Then you can extend them as long as you only ever append, which is a constraint on protobufs anyways. This is how most media containers have worked since the Amiga days.

Meanwhile, their encoding/decoding is infinitely faster than most other serialization formats, allowing you to serialize data back and forth to disk or as IPC faster than anything else.

Re:Ask HN: When does it make sense to roll your own X? by pjc50 on Sunday July 14, 2019 @11:09AM (#20433816)
.. as long as you never change endianness, structure packing, word alignment requirements, and always use types with explicit sizes. And you have to keep the data in the structure itself, so as soon as you hit a pointer you need to write explicit serialisation.
Re:Ask HN: When does it make sense to roll your own X? by CoolGuySteve on Sunday July 14, 2019 @11:17AM (#20433860)
> .. as long as you never change endianness, structure packing, word alignment requirements, and always use types with explicit sizes.

#pragma pack(1) and byteswapping solve all this and neither are more work than specifying some intermediate format and calling some external tool in your build system. In C++ you can make types that swap automatically if required.

> And you have to keep the data in the structure itself, so as soon as you hit a pointer you need to write explicit serialisation.

I have yet to see a serialization format that does this well and quickly. Just make a new message type for it that follows the first.

Re:Ask HN: When does it make sense to roll your own X? by makapuf on Sunday July 14, 2019 @11:25AM (#20433903)
Or juste use riff which useful and provide just enough to implement wav, bmp and other file formats (so you'll find heaps of examples(.
Re:Ask HN: When does it make sense to roll your own X? by Matthias247 on Sunday July 14, 2019 @11:43AM (#20434007)
Indeed. Using plain structs one can't just add a single field with a reasonable default and still be backward compatible - which is a very very common case in API design.

You could add completely new revision of the API. But having to maintain it - typically additionally to the old revision since both sides might not updateable in atomic fashion - is a lot of extra work.

Serialization which allows to add and remove fields in a backward compatible fashion is a mandatory requirement for any bigger sized project for me. If you need super high performance maybe use Flatbuffers or Cap'N'Proto instead of Protobuf/JSON/CBOR - but I really wouldn't recommend to go for plain structs.