It might not be viable, but I used to "publish" internally my own project one pagers, and update the timelines - any discrepancies simply became a shield for this future arguments.

In short it sounds like you don't have the power to say no - but actually you do.