Well, you could look at it that way. But the "encoding" is the same encoding that an "int" field in a struct uses, and "decoding" it is a trivial load instruction.

At what point is byte ordering handled?

The format is little-endian, so it's necessary to use a little-endian load instruction, which almost all architectures (including most big-endian architectures) have.

(But every widely-used arch these days is LE.)