That is decriminalizing, not legalizing. Even taking decriminalization, however, I would wager there are many, many more people who simply want to legalize cannabis relative to those who might want to decriminalize/legalize all drugs.

It is probably more accurate to say lots of liberals and conservatives want to legalize cannabis, and a non representative extreme contingent on both sides either want to decriminalize all drugs or continue to ban cannabis.