It's not what they are made of, it's the effect they have. Bits get stuck between your teeth and gums, and this encourages bacteria to grow and establish biofilms, which once established can start causing tooth decay. Long term, they can cause a lot of damage.