IANAL but you could likely successfully claim that you forgot where the key is, exactly because that's a product of the mind. If they have evidence that you do actually know then you might be compelled to hand it over, though.

If law enforcement has a warrant to search your safe, they could presumably expand that search to the rest of your house if you forgot where the key is. The core distinction is that the key is a physical object, it exists somewhere even if you forgot where it is. By comparison the combination is a product of the mind. The only way to retrieve it is for someone to talk to the police (which they have a constitutional right not to do).