To steelman this argument (to the extent that I can): we can in fact attribute a great deal of harm to highly processed ingredients, while there are a number of plants and animals humans have lived with and eaten for millenia, and we've identified the natural products that are poisonous. Weed is one of the plants that humans have coexisted with and consumed for millenia, and societies with high weed consumption don't appear to be significantly sicker than ones that don't.

(I can't think of a way to steelman around the fact that there are many toxic plant and animal substances, and some highly processed ingredients are completely harmless)