Yes, you're technically correct, but I can assure you that byte-swapping is fast.

It seems I should have just left out the fact that byte-swap is a single instruction on x86, I was trying to make the point that it is fast but I suppose the point missed its mark and the fact that it's a single instruction really doesn't matter. You can do the swap with a few shifts and it still takes no time at all. If speed really becomes such a factor that this is too slow doing the encoding and decoding, you'll likely be looking at replacing this framework with a more hardcoded solution, not removing byte-swaps.

1 or 2 uops, 1 or 2 cycles (depending on data size). So quite fast. There's also the newish MOVBE (load/store big endian), which currently performs fairly similarly to MOV + BSWAP per Agner, but could be made into a single uop in the future.