Abuse, violence, stealing and so on are already crimes in their own right, regardless of whether drugs are involved or not. They directly involve harm inflicted on other people. The act of doing drugs does not, so why make that a crime, too?

It's a slippery slope argument, it's like saying cutting yourself should be a crime, because people who cut themselves have issues and could harm others.

Just like drug users, they should be helped, not punished.