But if the big-endian case isn't run through CI then bugs will still creep in, where people forget to use the abstraction and just cast a buffer pointer to do a 32-bit read, or where they don't keep the distinction between 'a 32 bit value in host endianness' and 'a 32-bit value in host-independent endianness (eg network data, or pixel data)' straight and forget a conversion step. QEMU's testing on s390 and ppc64 catches a steady trickle of this kind of bug before it goes into the codebase.

(You can catch unaligned-access bugs on x86 with clang/gcc address-sanitizer, I think, but it can't help with endianness bugs.)

Your example is actually for the most favourable case because there are native 32-bit types. I've seen countless attempts to implement three-byte records via custom Uint24 classes that overload operators and abstract endianness (in various buggy ways) instead of just reading three damn bytes in a uint32_t. Supposedly in the name of readability ("I want to be explicit that this stores a 24-bit variable, not a 32-bit variable, so I'm going to copy-paste these 200 lines of either template magic or line noise, I'm not sure, but see how readable it is now? Oh yeah it's still 32-bit underneath, yeah, I'm not sure what happens if you write a 32-bit variable there, I think it breaks. This is way better than a typedef though, 'cause, uh, C++-17?)

    // wrong
    uint32_t num = ...;
    // num is set to 0x10203040
    num >> 24 == 0x10; // On little endian
    num >> 24 == 0x10; // On big endian

One thing I have to do frustratingly often is to implement a bunch of functions like "read_uint32_le", it is a pity that there is no comprehensive standard.

([1] On the other hand, I miss the good old days where you could treat C like "high level assembler", a pointer was just a number that indexed memory, and you could reinterpret bytes with impunity - sometimes had to to get decent performance!)

I wonder what the arch difference is between the clangs that do it and the ones that don't, I don't know much about arm and risc-v.

Not sure about RISC-V but I would not be surprised if the same applies there.

(Some instructions insist on aligned data, like atomic read-modify-write operations and some types of SIMD insn, but x86 has a few needs-alignment special cases too. Basic "just load/store data" insns that the compiler will emit are fine.)

It's probably a case of lowest common denominator.

Unalligned access is discouraged but allowed by the spec... But it's allowed to raise an exception. But not required. Cheaper to optimise to avoid that circumstance.

But for the longest time people would basically dump the raw bytes of their structs from / to the network whenever possible, e.g.

    struct my_data {
        int foo, bar;
        float x,y,z;
        long dontdothat;
    } data;

    write_bytes_to_the_network((const unsigned char*) &data, sizeof(my_data));

But yes, people often start that way and regret only when it is too late and the files are in the wild.

Imho a better way to handle this would be to use a macro or special function that comes with the compiler, assuming it exists of course.

If you know you are reading aligned memory and you need to do it with a single word memory access, then you need to use intNN_t* and use htonl or similar functions to byteswap. There’s no way around it.

  cpu_to_le32(),
  cpu_to_be32(),
  be32_to_cpu(),
  le32_to_cpu(),

These functions will either swap, or not, depending on the endianness of the target machine, and this decision is made at compile time, so the "no swap" case imposes no performance penalty.)

That way, the intent is clear, and the code works on big endian and little endian. So instead of what you propose, which I imagine you might use via:

  x = some_value; /* say this is big endian, and we want it little endian, because we're on Intel. */
  uint32_t y = read_int((char *) &x);

  y = be32_to_cpu(x);

Likewise, you could write:

  x = cpu_to_be32(y);

This is similar to htonl(), ntoh(), htons(), ntohs(). (host to network/network to host long/short).

Except it is. On x86, which is little endian, compilers are smart enough to convert it to a mov.

> These functions will either swap, or not, depending on the endianness of the target machine

So will the above. Here's the MIPS little endian version:

    read_int32(unsigned char*):
        lwl     $2,3($4)
        j       $31
        lwr     $2,0($4)

    read_int32(unsigned char*):
        lbu     $3,1($4)
        lbu     $2,2($4)
        sll     $5,$3,8
        lbu     $6,0($4)
        sll     $3,$2,16
        lbu     $2,3($4)
        or      $3,$5,$3
        or      $3,$3,$6
        sll     $2,$2,24
        j       $31
        or      $2,$3,$2

But it is. That's the whole point, to not bother with weird endian conversion macros.

You're telling me that what the linux kernel guys do is "weird endian conversion macros?" I don't believe you.

It doesn't always swap. Look at the assembly I posted above comparing mips64le to mips64be.

The code takes the byte at b[n] and shifts it (8.n)-bits. These are the same bytes on both little endian and big endian machines. They read the bytes in the same order (little endian), and convert them to the native order.

You can also see very clearly that it doesn't always swap by looking at the simple x86_64 assembly output that the was posted where no shifts or swaps are performed. Just a mov instruction.

Read the assembly. Or better yet, try the code out for yourself.

Assuming LE ordering on the writer side, you would need to swap again on a BE machine. Usually, however, you want to store in your target byte order, though. One could argue this is irrelevant as long as the memory doesn't leave your application, of course. Yet, by explicitly having macros _to_cpu, _to_be, _to_le and so on you make all of this explicit.