I think you should be worried instead of the amount of libraries that you give up from the JVM and Python ecosystems before anything else. Also, the applications of Rust and Java/Python generally do not overlap, I would be cautious in approaching this transition. Scala would be a better choice since you have expertise on the JVM and it has a familiar syntax to Python devs.