When you look back, you're looking back from the particular world where X became a requirement. If instead, Y had become a requirement, you would be looking back wishing you had structured your code in some completely different way. YAGNI then applies to the extent that you can't predict which of X and Y it will be in advance.
That's the trap that Magic and poker players call results-oriented thinking: You got burned once by making the best decision available given the information you had, and you give that experience a disproportionate weight, like the player that lost big with a pair of kings in his hand, and undervalues a pair of kings after that.