Yep that's exactly what you would do.

However, in practice, this usually is just not a good indicator of confidence. NNs are notoriously overconfident.