Refining is usually done to exclude other parts or to make a single issue product. Not bad on its own but bad if added in huge quantities. e.g. Refined sugar is usually the same a cane- or beet sugar only the source is different which generally makes it cheaper.

> "Many carbs are bad due to heavy processing and added sugars/fats, many are good such as fruits and vegetables."

Counter example: adding apple syrup is good? (Hint no to an extend: as it is mostly fructose and glucose, even if some nutrients remain)

In my opinion you fall into the oversimplification trap yourself.

And as with everything food related, moderation is good.