Caches don't break atomicity.
Hint: they're not, and that causes all kinds of subtle problem when programmers assume they are, especially when it comes to atomicity. The compiler caches a value in a register and two threads synchronize on that value? Boom. Or maybe no boom when one was expected.
The example cited was in a library written in C.
This is why it is important to tell the language what you mean. Even in C non-atomic access don't have these guarantees. It may seem pointless because in 99% of cases that "non-atomic" store compiles to the exact same instructions as a relaxed store. But that is just because you are getting lucky. The language doesn't guarantee that and with the atomic store the compiler is well within its right to emit something different (like two stores, spilling dirty values, ...).