This is absolutely what happens. I've been fortunate in my own management over the years but I have witnessed atrocious management "nearby" and talked to friends at other companies in similar situations. One or two quit entirely because of their direct managers. Layoffs happen, incredibly talented people are pushed out the door for some stupid reason or another, and yet the most poisonous of managers somehow keep their jobs at most companies it seems.