Relying on undefined behavior because your compiler happens to behave a certain way today on one specific machine is NOT the same as wrapping unsafe with safe. Next version of the compiler can exploit the undefined behavior and simply remove the code as an optimization, or even worse the optimization is already in your current compiler and only happens under some strange circumstances you are not aware of.

Unsafe code blocks still have defined and predictable behavior, but you have to guarantee the memory safety.