The imperativeness of Racket really doesn't come so much from the *! functions, but instead things like `begin` which explicitly sequence program statements in such a way that only allows them to interact via external mutable state.
I have a feeling we are on yet different definitions of imperative and such.
(define x 3)
(display x)
(set! x (add1 x))
(display x)
is about as imperative as they come, by my definition. And not a begin in site.
I was actively avoiding the obviously imperative functions, though. Not sure why. That is, I could have simply said that the existence of all of the functions that end in exclamation points shows the language is imperative.