See it this way: Numbers are distinct from their representations. 0x10, 0o20, 16 and 0b10000 are four different representations of the same number. The & operator doesn't operate on _numbers_, it operates on their binary representation — in many ways, it's almost like string manipulation.

What I suggested was that that way of operating on the representation of the number would work on any base in a way that is consistent (but, importantly, doesn't yield the same numerical result!) with the binary version.