Although it won't solve the general performance problems (it's a slow rocketchip impl) you should be using NBD root. That's what we use for the Fedora/RISC-V builders.