Putting the private key somewhere that antagonists could read it was insane. The fact that the public key was also there didn't do any extra harm, and removing the public key wouldn't help at all. Any attacker has (or can get) your public key, because anyone interested can, that's its purpose.