I also ran Windows Server 2003 as my regular desktop OS for a few years. I got an evaluation copy in the mail at some point and just kept using it. Even with the extra overhead of running network services on it for my household, it was still more performant than Windows XP on that same computer.
I ran it for awhile too, on my laptop. I think that was back in my post-Vista try-all-the-other-options-out-there days, as I cannot remember what advantage it gave me that XP didn't have, but I do remember it being a pain to get set up for DirectX and other things that you generally wouldn't use on a server. Still, it did look cool to see my laptop displaying its "Windows Server 2003" screensaver in the library.
To get the best of both worlds I ran a spare copy of Windows Server 2003 as my "daily driver" on a Thinkpad back in the early 2000's. It had the Windows 2000 UI with the behind-the-scenes updates that came with Windows XP. Most software that installed on XP installed fine on Server 2003. It was nice.