When thinking of segmented architectures my mind jumps to 16-bit x86. Of course those are integers. It's common to store pointers in a 32bit value where half is segment and half is offset. A comparison of linear addresses would take segment * 16 + offset. No need to use the segment for that calculation.

Obviously it could be different for something more exotic, but I had a hard time seeing how it wouldn't eventually lead to or map well to integers.