> Casting an integer to a pointer is implementation defined, not UB.

You're right, that was a bad example. Here's a better one:

    int x, y;
    ptrdiff_t diff = &x - &y;

This is Undefined Behavior, because &x and &y don't point to the same object.