Yeah, I can't imagine that scenario at all. In fact, Obama publicly refusing to attend Trump's inauguration would have probably been met with thunderous applause from the "not my president" crowd.

Maybe if he had declined to attend and refused to recognize the results of the election by falsely claiming fraud and spreading misinformation? And encouraged people to storm the capitol? Maybe then it’d make sense for Obama to have been banned.