I did play Forza Horizon 4 on the PC with a Xbox Series X controller and it worked fine. But had bad experience if I tried to use an PlayStation 4 controller. Which controller did you use?
But for Forza Horizon 5 I do play it on a Xbox Series X and currently prefer it for two reason: 1.) Its just works without any major issues (a friend of mine has on the PC currently constant crashes) 2.) My PC hardware can run the game but not pretty well
One big benefit I see on the Xbox side is that you can play the games on your PC too. And I hope that "exlusives" die as its just annoying.
But in short if you just want to play games a console might be less friction compared to a PC. No need to fiddle with graphics settings and so on.
I use PS4 controllers with my PC(linux), pretty sure they work with windows too. What controllers cannot be connected to a PC?
But the main reason for me is the controller: It's just so much better than any other controller I've connected to a PC. The back triggers have excellent force feedback and in some games the vibrations tell you on what type of surface you're walking. This might not count for XBox users, which can use the controller on their PC.
Also, I use a Mac, and I don't want to bother with setting up a gaming only Windows PC. If I factor in time spent, $500 is worth it.