This puts them in a double-bind: do they tell you to work on Y and come off looking like an asshole to person A? If it truly is important (or they are assholes), they'll say yes.
Then it's a simple matter of telling person A that you'll be behind on their project and letting politics take its course. If project X is really important to person A, then they'll go talk to person Y. Either way you're in the clear. Once you've gotten another person involved, it's much more difficult for someone to gaslight you in the manner you speak of.
Unless you're the kind of employee that needs constant hand-holding and micromanagement, you're still responsible for your other tasks. If X will slip because of the new work, it's up to you to communicate that clearly to those concerned.
Sure, some people expect the boss to anticipate and keep track of all of that. Usually the same people that bitch about not being given enough responsibility, freedom and trust...
In the end, it all depends on communication: if you don't clearly agree on what will happen to X and Y, both you and your boss will be screwed. Assumption is the mother of all fuck-ups.
The micromangament thing goes both ways: Perhaps you're the boss that doesn't trust their people to do it right [implicitly or explicitly], or create a situation for your employees where everyone is constantly fighting fires (short-term projects) instead of tending to mid to long term objectives. (When "mid term objective" might even mean "a week from now"... depending on how bad it is).
So, even if you're not an employee who needs micromanagement, you may get it.
Then, someone remembers about the X project, suddenly, and it's a big emergency (and probably the employee's "fault", because it "was on your list of responsibilities".
I agree that communication should be clear, but even when it is sometimes you get called on the carpet for not doing that task you were not given time to do. (Although another commenter suggested getting things in writing... which might help.)
> Ask what you should do first. Should you stop working on X and Y and finish this new project first?”
How... niave. In my experience, in these situations the boss will say, "Work on Y"... then when X is due, wonder why X isn't done (and blame you).