I may misunderstand something here, but I'd assume the organizers of a conference are in charge of deciding which talk goes where. Why is there an expectation for this to be different here?

If I was askwd to give a keynote somewhere and upon presentation of my topic the organization decides another talk is better suited I would trust them that they wouldn't do that lightly.