Yes, if you want FIPS 140-2 you are going to pay the big bucks. Yubico's Yubikey 4 has a FIPS certified version so they have been through the process with other products but you are right that their HSM is not certified. If you care about FIPS certification you can look up if a product is certified or not in NIST's database. [0]

[0] https://csrc.nist.gov/projects/cryptographic-module-validati...