I've stood up for someone in that kind of conversation. Our boss was yelling at a colleague in an inappropriate way. I interrupted to defend the guy. Our boss said, "If you don't feel like you need to be here, you should leave the room." I left the room. No adverse consequences to my job, as far as I can tell.

I suppose I could have stayed, but I also figured my colleague could stand up for himself. He's not a pushover.