Check out HTJ2K if you haven't already, it's J2K with SIMD instructions, it will have solid support because the medical industry is (slowly) moving towards adopting it. Sounds like you already saw NVIDIA's J2K decoder. There are FPGA solutions as well that are insanely fast (CAST and some others).

HTJ2K has support in OpenJPEG (master branch at least), might need to compile with a specific flag or something I can't remember.

I think JPEG-XL is ideal for these types of applications in my opinion, even if you ship the decoder with your client. Their progressive encoding/decoding is much better than J2K in my opinion.