I think it is wise to publicly announce both speakers who have accepted invitations and speakers whose invitations were revoked (don't even have to say they are revoked, simply stating "unfortunately John Doe will not be speaking in our conf" should be sufficient). If not for these situations, then simply to inform visitors whom they can expect to hear in the conference.