No downsides I've experienced. I switched over 6 months ago, didn't change my .password-store directory at all really, and everything has been smooth. There was even a bug a little while back I reported that was fixed pretty quick.

I had to switch over to support a team who were using gopass too for secret sharing, otherwise I'd probably just be on pass still.