Re:Add extra stuff to a “standard” encoding? Sure, why not
by saghm on Wednesday September 20, 2023 @04:29AM
(#37581818)
> 4 bytes is actually too long for short messages
Would it ever be an actual bottleneck though? If it's not actually impeding throughput, I feel like this is more of an aesthetic argument than a technical one, and one where I'd happily sacrifice aesthetics to make the code simpler.
> some message can exceed 2^32 bytes
Fair enough, but that just makes the question "would 8 bytes per message ever actually be a bottleneck", which I'm still not convinced would ever be the case
Re:Add extra stuff to a “standard” encoding? Sure, why not
by lifthrasiir on Wednesday September 20, 2023 @06:05AM
(#37582390)
I agree that 4 byte overhead in a single item is not a big deal even when the item itself is short. But if you have multiple heterogeneous items, it becomes worthwhile to set an upper bound for the relative ratio of overhead over data. (SQLite, for example, makes extensive use of varints for this reason [1].) I personally think a single byte overhead is worthy for sub-100-byte data, while larger ones should use a fixed size prefix because the overhead is no longer significant.
Re:Add extra stuff to a “standard” encoding? Sure, why not
by catiopatio on Wednesday September 20, 2023 @06:19AM
(#37582474)
I work on an embedded OS where our IPC buffer is only about 140 bytes.
Anything more requires multiple IPCs, with lots of expensive context switches.
Wasting even one precious byte on a pointless header would absolutely be an issue in this environment.
Re:Add extra stuff to a “standard” encoding? Sure, why not
by saghm on Wednesday September 20, 2023 @03:33PM
(#37588935)
That makes sense. I've probably been thinking more about remote network protocols, where the time it takes for a message to reach its destination will be so large that the overhead for a few extra bytes per message is negligible; for a protocol used solely within a given local system, there would be a much lower threshold of acceptable overhead.
[1] If you ever have to design one, make sure that reading the first byte is enough to determine the number of subsequent length bytes.