You can't use that argument and have turing completeness at the same time.

Yes, you can. You can emulate a Turing machine (except with a finite memory, of course) in safe Rust, but given that the Turing machine is entirely serial by design, I don't see how is that even relevant.

In practice, Rust has `unsafe {}` escape hatch that allows building of abstractions (like mutexes) for the safe majority of Rust using contained bits of as-safe-as-C code where necessary.