What breaks are
- truly dubious programs which use pointers in ways they really shouldn't, and are almost certainly just bugs; like use-after-free. When a pointer with an out-of-date tag is passed to the library, that is pretty much a confirmed use-after-free, which is not a legitimate idiom of non-maximally-portable C, like assuming that all pointers are the same size.
- C programs which manipulate the pointer representation: e.g. run-times for dynamic languages that put their own tags in pointers. These can easily work around tagging. (I've dealt with this on Android recently, quite easily. For the affected objects, I strip the tag away, and work with the untagged pointers. When it's time to pass the pointer back to Android's library, I put the tag back. All other code works as before.
(C++’s std::aligned would be less awkward, but, well, memalignment() is what we get.)
You still can’t portably store pointers and integers (or doubles) in the same place and distinguish them, though.