The Byte Order Fiasco

Re:The Byte Order Fiasco by raphlinus on Saturday May 08, 2021 @09:30AM (#27086957)

The idiomatic way to do this in Rust is to use functions like .to_le_bytes(), so you have the u32 (or whatever) on one end and raw bytes (something like [u8; 4]) on the other. It can get slightly tedious if you're doing it by hand, but it's impossible to accidentally forget. If you're doing this kind of thing at scale, like dealing with TrueType fonts (another bastion of big-endian), it's common to reach for derive macros, which automate a great deal of the tedium.

[ Parent | on HN ]

Re:The Byte Order Fiasco by nly on Saturday May 08, 2021 @09:57AM (#27087163)

Who decides what methods to add to the bytes type/abstraction?

If I have a 3 byte big endian integer can I access it easily in rust without resorting to shifts?

In C++ I could probably create a fairly convincing big_uint24_t type and use it in a packed struct and there would be no inconsistencies with how it's used with respect to the more common varieties

[ Parent | on HN ]

Re:The Byte Order Fiasco by raphlinus on Saturday May 08, 2021 @10:27AM (#27087411)

In Rust, [u8; N] and &[u8] are both primitive types, and not abstractions. It's possible to create an abstraction around either (the former even more so now with const generics), but that's not necessary. It's also possible to use "extension traits" to add methods, even to existing and built-in types[1].

I'm not sure about a 3 byte big endian integer. I mean, that's going to compile down to some combination of shifting and masking operations anyway, isn't it? I suspect that if you have some oddball binary format that needs, this it will be possible to write some code to marshal it, that compiles down to the best possible asm. Godbolt is your friend here :)

[1]: https://rust-lang.github.io/rfcs/0445-extension-trait-conven...

[ Parent | on HN ]

Re:The Byte Order Fiasco by nly on Saturday May 08, 2021 @10:47AM (#27087600)

I agree then that in Rust you could make something consistent.

I think there's no need for explicit shifts. You need to memcpy anyway to deal with alignment issues, so you may as well just copy in to the last 3 bytes of a zero-initialized, big endian, 32bit uint.

https://gcc.godbolt.org/z/jEnsW8WfE

[ Parent | on HN ]

Re:The Byte Order Fiasco by raphlinus on Saturday May 08, 2021 @11:29AM (#27088009)

That's just constant folding. Here's what it looks like when you actually need to go to memory:

https://gcc.godbolt.org/z/9qGqh6M1E

And I think we're on the same page, it should be possible to get similar results in Rust.

[ Parent | on HN ]