HSM for the hot wallet probably provides greater security than no HSM for the hot wallet but I don't think it gives you that much extra security.
> Amazon says they don't have access to your HSM but presumably they do if they wanted to.
I wouldn't be so sure. As far as I can tell, once you've provisioned an HSM from Amazon you have full administrator access. That includes control over all trusted SSH certs, users, etc. Of course Amazon has physical access, but that doesn't give them access to the HSM's crypto functions or the key material.
If Amazon is a bad actor they trivially have access to the HSM because they could just write some software that pretends to be the HSM. However, this is probably not in the threat model because the amount Amazon loses by fucking a client like this is much more than the amount they would gain by fucking a client. The real thread is rogue Amazon employees and I guess it would be be hard for them to MITM the HSM from the start.
Your only protection for the hot wallet is to obscure the credentials you use to connect to HSM. Good luck with that against and a determined adversary that has a lot to gain from dumping the HSM key. (if it is not clear this 'only protection' thing is when the adversary has root access to your machine)
Oh.. and if you are a gemini dev. Obscure your code encrypt your HSM credentials in memory. :) Security through obscurity is actually a useful thing against attackers.
> If Amazon is a bad actor they trivially have access to the HSM because they could just write some software that pretends to be the HSM.
I believe SafeNet HSMs protect against attacks of this nature.
> This manufacturer-validated devicee identification mechanism enables a strong trust model whereby customers can be assured that they are communicating with specific SafeNet hardware units in a way that cannot be spoofed.
I'm a little disappointed that they only have level 2[1] HSMs in the cloud, as I would be uncomfortable protecting my hot wallet keys with only tamper evident protections, rather than level 3+ that actually attempt to detect intrusion and delete keys. Bitcoin makes for very quick stealing once you have keys, so reactive defenses against key loss don't help much as you're literally in a race condition with the attacker to empty the wallet (you into a non-compromised one, the attacker into their own). But I would assume they weighed cost/risk and I've never heard of a security compromise of Amazon's HSMs so it was probably a reasonable choice.
edit: I should also applaud their use of PGP and (explicit) respect for responsible disclosure.
[0]: https://exchange.gemini.com/security
[1]: https://en.wikipedia.org/wiki/FIPS_140-2#Level_2