If I remember 16 bit C compilers let you choose a couple of different memory models. I remember 3, data and code fits in 16bits. data is 16 bits and code is segment and offset. And both pointers are segment and offset. I think there were more, especially for systems with 'high memory'
Most processors a pointer just gets stuffed into a register and then that's used by a assembly instruction to read/write a memory location. But it's certainly possible to have systems where it's not so simple.
But it is possible to imagine completely exotic CPU architectures, where there is a separate type of register that can only be used for pointers and it is impossible to convert to int; you could still write C compilers for that architecture and be 100% conforming to the standard.