This looks really nice as a hobbyist project. That being said - I can't see this being usable in an enterprise space. There's no TPM requirement so someone able to get access to the HSM could dump the keys from memory.

The worst part of using HSMs in enterprise env's is when you're forced to use something certified for FIPS 140 Level 2 or 3 requirements.. Then you're paying for what amounts to a shiny foil "anti-counterfeit / tamper-evident" sticker. This could probably easily meet level 1 via a certified openssl module, but alas, with most enterprise HSMs you're not paying for the _HSM features_.