The definition of harm is context-dependent. I think you mean that opiates and other drugs, addictive or not, when “taken at a safe dose in a health young adult” will not significantly degrade basic body function.”

But opiates taken over a relatively short time can and often do greatly impair CNS neurotransmitter systems and downstream behaviors—-in some cases causing almost irremediable dysfunction. This happens to a sizable subpopulation of humans; more than 10% of physician-prescribed oxycontin users. Genetics and social context modulate vulnerability.

This is definitely not “little harm”.