My Switch's CEC is as flaky as yours, but my Xbox One X does wake up the TV from sleep no worries even if I let it go to sleep, last I checked. Though now I really want to double check this, and see whether it's the Xbox that has messed it up, or the Xbox + TV manufacturer combo?

My Apple TV 4K wakes my TV and to the right input with Airplay though!