I'd clarify and say it's the brand library. I bought an Xbox One because I wanted to play the Halo series, which I knew I couldn't do on PS4. The same might apply if you were really into Gears of War or Forza.

Otherwise, most people who "switch" consoles keep the legacy console to play "old" games rather than shelling out for the same content twice. Particularly now in the age of account-specific digital downloads that you can't sell / recoup on the secondary market.