IIRC, the biggest drug on the Silk Road was marijuana, which was probably not cartel grown. There are plenty of places in the US where it's effectively legal to grow marijuana, and the growers who have a legal enterprise almost certainly produce plenty on the side for the black market.

Cocaine and heroin are different stories, but are probably the exception compared to marijuana, various drugs that need to be lab-synthesized, and prescription pills being resold.