They are optimized, yes, but from what I recall from reading the JVM code a few years ago, some optimizations don't get applied to those reads/writes. For example, summing two arrays together will be vectorized to use SSE instructions while doing so through Unsafe won't [0].

[0] https://cr.openjdk.java.net/~vlivanov/talks/2017_Vectorizati...