tch-rs is really nice. I have built a Rust sequence labeler + dependency parser + lemmatizer on top of it. It supports multi-task learning, finetuning of BERT-like models, model distillation, etc.:

https://github.com/stickeritis/sticker2/

Unfortunately, when I started this guillaume-be's rust-bert crate wasn't around yet (I think), so I also ported some of the models from Huggingface transformers:

https://github.com/stickeritis/sticker-transformers/

At any rate, I can highly recommend the tch crate if you are looking to build neural networks in Rust.